QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching

Key points

• In this work, we establish a systematic model of MLA's dual-path quantization errors, characterizing their distinct effects on attention-output distortion and explaining the pronounced amplification of RoPE-path errors.1
• Guided by this analysis, we introduce QuantMLA, a function-aligned framework for low-bit dual-path quantization.1
• Within these spaces, QuantMLA learns path-specific transformations with function-aligned objectives: attention-output reconstruction captures the content path's coupled matching and aggregation errors, while positional QK reconstruction preserves the RoPE-induced component of the attention logits and admits a theoretical bound on output distortion.1
• We develop a native low-bit MLA attention kernel that integrates unpacking and dequantization directly into attention computation.1

Sources (1)

• [1]QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching
  Hugging Face Daily Papers · Sep 30, 12:00 AM

  Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoupled RoPE paths, yet cache memory still scales linearly with context length and batch size.

  In this work, we establish a systematic model of MLA's dual-path quantization errors, characterizing their distinct effects on attention-output distortion and explaining the pronounced amplification of RoPE-path errors.

Extractive summary: sentences quoted from the sources.