QLoRA on ExLlamaV3 Quants - Train @ 2-8bpw
A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without transformers.
Proof1 community thread
Key points
- With it, you can train 2-8bpw exl3 quantized models, rather than the old BNB 4 bit lock-in.
- EXL3 models are currently the highest precision models for their size, which leads to lower loss.
- Llama3, Mistral Small/Medium, Qwen3 dense and MoE, Qwen3.5/3.6/3.8 hybrid models, Gemma3/4 including MoE, Arcee Trinity.
- Real-time training, where the loaded model serves chat and trains its adapter in place
Sources (1)
- [1]QLoRA on ExLlamaV3 Quants - Train @ 2-8bpwr/LocalLLaMA (top, daily) · Oct 12, 07:30 AM
A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without transformers.
With it, you can train 2-8bpw exl3 quantized models, rather than the old BNB 4 bit lock-in.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026What is Decision 3.0? vLLM Semantic Router’s New Open Decision Models
- Oct 11, 2026Gemma4 E4B EXL3 - Alternative to Llama.cpp on Jetson Orin
- Oct 11, 2026Every Model That Can Be Run On 10-16GB VRAM Ranked
- Oct 9, 2026Microsoft's Decision-1 model enters the fast-growing AI decision model race
- Oct 9, 2026Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash
- Oct 8, 2026huggingface/trl v1.15.0
