Metraipolis › archive › dc_5aaf07baabb0

Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families — arXiv:2609.16391

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.16391
License (as archived)
CC0-1.0
Archived
2026-10-15T19:23:26.941Z
SHA-256 of the archived text
eef64b4a06032a294e09ea6e6f044b3f5579f0f5d4695f4895209f41b61624e4
Stable URL of this page
https://metraipolis.com/documento/dc_5aaf07baabb0

How to cite

Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families — arXiv:2609.16391 — arXiv (en), https://arxiv.org/abs/2609.16391 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5aaf07baabb0 (Metraipolis archive dc_5aaf07baabb0, sha256 eef64b4a06032a294e09ea6e6f044b3f5579f0f5d4695f4895209f41b61624e4)
BibTeX
@misc{dc5aaf07baabb0,
  title = {Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families — arXiv:2609.16391},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.16391},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5aaf07baabb0; sha256 eef64b4a06032a294e09ea6e6f044b3f5579f0f5d4695f4895209f41b61624e4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16391. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact. We test that advice on retrieval embedders directly, quantizing five checkpoints from four architecture families across a grid of bit widths and group sizes, and isolating the embedding, attention and feed-forward blocks at each width. Every heuristic fails to transfer as stated. The embedding table never emerges as the dominant isolated protection priority in any family, despite being the largest tensor in several of them. Module sensitivity does not survive as a transferable ordering: at INT4/g16 the spread between modules is too small to allocate against, at INT3 the ordering becomes family-dependent and joint damage stops being the sum of its parts, and at INT2 comparable reconstruction error accompanies retention ranging from 1.3 to 65.9 percent of full precision. A cheap reconstruction proxy is useful for screening uniform bit widths but substantially less reliable for choosing which tensors to protect; its apparent strength across the whole grid is a range-extension artifact. A distilled 109M student at INT3 holds 78.04 NDCG@10 in 68.4 MB and dominates the extreme-PTQ arm of its own 0.6B teacher, 297.9 MB at 64.46, on both size and quality -- but only inside the task it was distilled for. Sizes are byte counts of files that exist rather than arithmetic estimates, and the measurement repository carries the byte provenance for every one of them.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL).
Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.16391 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:32:32.898Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).