Explore
Everything AION read, in seven sections. Pick one, a topic or a time window.
A/B Blind Taste Test - Krea2-Turbo vs. Qwen-Image 2.1 Turbo
Since Qwen 2.1 Turbo dropped suddenly right after my last taste test, figured might as well do one for it as well.
I ran Qwen 3.8 Flash-Next on my AMD 7900 XTX at 500k context. All local and what a shift it has been.
I was running Qwen3.8-Flash-Next (base and later Swift-1.5 NVFP4) on a paid pod seat for about a month, and it started answering with short strings that echoed its own earlier reasoning.
VideoSame Model, Different Speed: Why Your Inference Provider Matters — FriendliAI
Open-weight models are good enough now.
VideoWhy DeepSeek Wants AI To Forget
📝 The DeepSeek OCR paper is available here:
What is Decision 3.0? vLLM Semantic Router’s New Open Decision Models
The vLLM Semantic Router team has released Decision 3.0, a family of multimodal decision models.
BerriAI/litellm v1.105.0
Verify using the pinned commit hash (recommended):
I tested different Qwen 3.8 27B quants
There is a lot of discussion which quant to use.
Every Model That Can Be Run On 10-16GB VRAM Ranked
It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features.
"Strata" for GLM5.3 Flash is here for some! Project Maya
I stumbled across this as I was currently having glm5.3 flash run only around 10tok/s basically unusable.
vllm-ascend updates (GLM5.3-flash) on dual 310p Ascend cards
Alright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less verbose me.
VideoAgents That Own Their Inference — Du'an Lightfoot & Khaja Omer, Akamai Technologies
Speculative decoding drops a demo model from roughly 58 tokens per second to 16 instead of speeding it up.
UPDATE: Qwen 3.8 27B 140 tok/s on single RTX 3090 Megakernel: KL divergence 0.0009 vs llama.cpp
RECAP: The megakernel is a CUDA engine for Qwen3.8-27B that runs 1.4-1.9x faster than llama.cpp on a single 3090.
qwen 2.1 turbo or normal - fixed the artifacts
hi guys, i fixed the confyui workflow with sigmas recommended by qwen 2.1 so artifacts are very reduced or no more...
Updated ComfyUI-SeedVR2-VideoUpscaler-with-TensorRT v1.6.6 - Native ConvRot W4A8 Support & DisTorch2 Scope
The other day, when I published information about v1.5.7, I received a request in the comments regarding support for w4a8.
I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens
Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and hashed n-gram embeddings, trained with a whooping budget of 100€
Now you can grow Bonsai on your potato
Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU)
BerriAI/litellm v1.104.3
Verify using the pinned commit hash (recommended):
Qwen 3.8 27B Q5 vs Qwen 3.8 Next Q3_S for document analysis
So far I've been using Qwen 3.8 27B Q5 with a 150K context window, but I'm wondering whether I should switch to Qwen 3.8 Next Q3S, since it has much more knowledge and could extract data much better than the 27B.
Converting dense models into Mixture-of-Experts
For the past few weeks I've been trying out converting existing dense models to sparse Mixture-of-Experts models, with no pretraining from scratch.
Reminder: try probabilistic MTP if you missed it. Decode +14% on prose
Optimal draft-n-max / draft-p-min seem to be in line with greedy sampling.
Running Next Flash IQ3_XXS at ~70 tok/s with 100k context or 2 instances of Qwen 3.6 35B A3B IQ4 at ~145 tok/s with 256k all on $500 of ex mining BC-250 boards
This will be my third update on the bc-250 cluster and for my first forray into local ai I have been having a blast.
Heretic or Abliterated?
Looking for recommendations for local LLM to convert an image into a usable prompt for Krea2 with max accuracy.
M5 Ultra 256GB or 2x DGX Spark? Which one?
Currently contemplating adding either M5 Ultra 256GB or dual DGX Spark in addition to existing 5090.
M5 Ultra Mac 256GB Studio - 64 core vs. 80 core version, is the inference speed difference worth the extra 10 weeks wait time?
I’m seeing 6-7 week delivery wait time for M5 Ultra Mac Studio 64 core version versus 15-17 weeks for 80 core version.