Final revision of zzpanic vllm-radiance for 1 x R9700 GPU, qwen3.8-27b
Since StillDeadcode has released his new radiance inference engine, the writing is on the wall for the old vllm-radiance builds, people have moved on to newer and better.
Proof1 community thread
Key points
- I can see a few more single digit percent speedups available but probably won't chase them.
- This borrows from quite a few people who have done work in the Launch80 Discord and is by no means my sole work.
- startup caching for vllm to prevent recompiling various things on each restart
- a working kv-offload feature for vram-sysram-disk, with memory thinning to save storage.
Sources (1)
- [1]Final revision of zzpanic vllm-radiance for 1 x R9700 GPU, qwen3.8-27br/LocalLLaMA (top, daily) · Oct 11, 08:47 AM
Since StillDeadcode has released his new radiance inference engine, the writing is on the wall for the old vllm-radiance builds, people have moved on to newer and better.
I can see a few more single digit percent speedups available but probably won't chase them.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026DeepSeek-V4.1 Flash: The Most Insane Optimization So Far
- Oct 10, 2026vllm-sr/d3
- Oct 9, 2026Microsoft's Decision-1 model enters the fast-growing AI decision model race
- Oct 9, 2026S$^3$R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding
- Oct 9, 2026Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash
- Oct 8, 2026ConwayResearch/Underdog-Saluki-27B-1.0
