Metraipolis › archive › dc_b84ad72ec4ea

QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization — arXiv:2609.00224

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.00224
License (as archived)
CC0-1.0
Archived
2026-10-17T06:32:52.466Z
SHA-256 of the archived text
d1d185a2ce355f7f70addad66b54f6d0746ffaf249aa3e40541399c4d2e73a8a
Stable URL of this page
https://metraipolis.com/documento/dc_b84ad72ec4ea

How to cite

QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization — arXiv:2609.00224 — arXiv (en), https://arxiv.org/abs/2609.00224 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b84ad72ec4ea (Metraipolis archive dc_b84ad72ec4ea, sha256 d1d185a2ce355f7f70addad66b54f6d0746ffaf249aa3e40541399c4d2e73a8a)
BibTeX
@misc{dcb84ad72ec4ea,
  title = {QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization — arXiv:2609.00224},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.00224},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b84ad72ec4ea; sha256 d1d185a2ce355f7f70addad66b54f6d0746ffaf249aa3e40541399c4d2e73a8a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00224. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existing PTQ methods often fail to generalize across models and suffer severe accuracy loss below 2 bits. Many leverage unstructured sparsity to mitigate this loss, but at the cost of regularity and GPU-friendly execution. We present QTEA, a sub-2-bit PTQ framework that quantizes weights into ternary values and uses salient weights as residual error compensators. To maintain hardware efficiency, residuals are assigned to selected columns with semi-structured $1:4$ sparsity within the salient columns. We further add column-wise rescale refinement to GPTQ-style column-by-column quantization, alternately updating per-column scales and ternary assignments to reduce reconstruction error. We also identify order-dependent error propagation in GPTQ and introduce error decay to attenuate late-stage error accumulation. On Qwen3-14B, QTEA compresses all weights to an effective 1.7 bits per weight while improving average accuracy over the strongest ternary PTQ baseline by 16.7%. It also achieves 1.40$\times$ and 2.61$\times$ lower perplexity on WikiText and C4 respectively. This trend holds on Llama3-8B, where QTEA obtains a 6.6% accuracy gain and 1.34$\times$ / 1.95$\times$ lower perplexity on the same datasets. Finally, we develop a lookup-table based kernel that achieves 7.2$\times$ faster per-token generation over an FP16 baseline. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.00224 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:15:12.755Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).