Running unsloth/Qwen3.8-27B-UD-IQ4_XS 262k q4_0 KV + vision on 16GB VRAM @ 80t/s and 1500pp
I get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card.
Proof1 community thread
Key points
- I am saddened whenever I see fellow 16GB VRAMers stuck on 60k context running q3 or lower quants or running weaker models because they think 27B it is out of their reach.
- I love this fork and by default it only supports q80/q40 but I modified it for my own purposes to support q40.
- But you can still do q80/q40 if you lower the context a bit or reduce the batch and ubatch to 128 or drop the vision maybe and increase kv-stream-arena-mib.
- As for the frontend, don't worry about that, that's just my launcher I vibe coded. llama.cpp-adaptive-kv-streaming is command line just like llama.cpp
Sources (1)
- [1]Running unsloth/Qwen3.8-27B-UD-IQ4_XS 262k q4_0 KV + vision on 16GB VRAM @ 80t/s and 1500ppr/LocalLLaMA (top, daily) · Oct 12, 08:14 PM
I get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card.
I am saddened whenever I see fellow 16GB VRAMers stuck on 60k context running q3 or lower quants or running weaker models because they think 27B it is out of their reach.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026unslothai/unsloth v0.1.906-beta: Browser Use + Qwen-Image-2.1-Turbo
- Oct 11, 2026I ran Qwen 3.8 Flash-Next on my AMD 7900 XTX at 500k context. All local and what a shift it has been.
- Oct 11, 2026I tested different Qwen 3.8 27B quants
- Oct 11, 2026UPDATE: Qwen 3.8 27B 140 tok/s on single RTX 3090 Megakernel: KL divergence 0.0009 vs llama.cpp
- Oct 10, 2026Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernel
- Oct 8, 2026unslothai/unsloth v0.1.905-beta: Sandboxing is here!