Metraipolis › archive › dc_ce0e9958f308
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary — arXiv:2610.21808
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21808
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:04:04.844Z
- SHA-256 of the archived text
9fa2f8a039b4aa6c69ed35900f3a4bce949eed5473dae1337778cf5011a409d4- Stable URL of this page
https://metraipolis.com/documento/dc_ce0e9958f308
How to cite
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary — arXiv:2610.21808 — arXiv (en), https://arxiv.org/abs/2610.21808 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce0e9958f308 (Metraipolis archive dc_ce0e9958f308, sha256 9fa2f8a039b4aa6c69ed35900f3a4bce949eed5473dae1337778cf5011a409d4)
BibTeX
@misc{dcce0e9958f308,
title = {RoleQ: Role-Aligned Scales for Converting Language Models to Ternary — arXiv:2610.21808},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21808},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce0e9958f308; sha256 9fa2f8a039b4aa6c69ed35900f3a4bce949eed5473dae1337778cf5011a409d4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21808. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy. Ternary weights have mostly been obtained by training from scratch or long fine-tuning. We introduce RoleQ, a quantization framework that places scales by the functional roles of Transformer units rather than by the layout of the weight matrix, as conventional formats do. Vectors that read from the residual stream take one scale per output unit; those that write to it, one per input unit. The rule is structural, needs no search, calibration or stored state, and holds on integer and ternary grids. In a rotated basis it matches a per-layer axis search at half the solver work, and at ternary it cuts perplexity by up to half against layout-based placement in two families at every size from 0.6B to 14B, where per-row scales leave MMLU at chance. Our lightweight post-training pipeline converts an existing checkpoint to 1.59 bits per weight in a few GPU-hours, a fraction of the cost of learned ternarization, and leads every post-training ternary method at matched or lower memory-resident bits on perplexity, zero-shot accuracy and MMLU, most widely on knowledge. A sparse second ternary plane placed by a per-unit Fisher saliency keeps the matmul multiplication-free and recovers what ternary loses first: at 8B, a fraction of a bit returns about half the MMLU that conversion costs, well ahead of random or Hessian-based placement at equal bits. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.21808 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.795Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).