Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders

Key points

• We measure a label-free substitute: quantization-induced representation drift, obtained by quantizing one module, re-encoding the corpus, and recording how far the output embeddings moved from their full-precision positions.1
• What is specific is the observable: the deployed output representation a dense retriever ranks with.1
• Across five development embedders, configuration-level drift orders sampled mixed-precision plans against held-out retrieval quality at a macro Spearman of 0.911, the sensitivity transports across calibration corpora and retrieval domains in the usable regime, module drifts compose rank-consistently but not numerically, and relevance-derived sensitivity adds no consistent value.1
• Output drift is thus a robust coarse sensitivity signal, not a universally optimal allocation objective: it avoids the catastrophic failures of the transferred signed-geometry adaptation and can remain usable at stressed budgets where uniform collapses, but fine-grained redistribution around a strong uniform operating point remains unresolved.1

Sources (1)

• [1]Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 11:44 PM

  Mixed-precision post-training quantization needs a per-module sensitivity signal; for a text embedder the obvious one -- the retrieval quality a module costs when quantized -- needs relevance labels that deployments rarely have.

  We measure a label-free substitute: quantization-induced representation drift, obtained by quantizing one module, re-encoding the corpus, and recording how far the output embeddings moved from their full-precision positions.

Extractive summary: sentences quoted from the sources.