Metraipolis › archive › dc_77d8d78ec4e5

TQLite: Multi-LLM Jury Guided Distillation for Real-time MQM Translation Quality Evaluation — arXiv:2608.02975

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02975
License (as archived)
CC0-1.0
Archived
2026-10-31T12:18:25.384Z
SHA-256 of the archived text
d6ca6332359398b457466d12ff9ae758a6e50e10f8c04fa5f6572db069efcfba
Stable URL of this page
https://metraipolis.com/documento/dc_77d8d78ec4e5

How to cite

TQLite: Multi-LLM Jury Guided Distillation for Real-time MQM Translation Quality Evaluation — arXiv:2608.02975 — arXiv (en), https://arxiv.org/abs/2608.02975 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_77d8d78ec4e5 (Metraipolis archive dc_77d8d78ec4e5, sha256 d6ca6332359398b457466d12ff9ae758a6e50e10f8c04fa5f6572db069efcfba)
BibTeX
@misc{dc77d8d78ec4e5,
  title = {TQLite: Multi-LLM Jury Guided Distillation for Real-time MQM Translation Quality Evaluation — arXiv:2608.02975},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02975},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_77d8d78ec4e5; sha256 d6ca6332359398b457466d12ff9ae758a6e50e10f8c04fa5f6572db069efcfba},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02975. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) have demonstrated impressive performance in MQM-based translation quality (TQ) evaluation, and recent advances in large reasoning models (LRMs) promise even greater improvements. However, both LLMs and LRMs are computationally expensive to deploy at scale, while small language models (SLMs)---though much more efficient---struggle with the complex reasoning required for evaluation tasks. In this work, we present an extensive empirical study benchmarking SLMs, LLMs, and LRMs across a wide range of TQ evaluation setups, providing a comprehensive view of the current landscape and establishing best practices. To address the scalability challenge, we introduce TQLite, a novel distillation framework that enables SLMs to approach the MQM evaluation performance of the best LRM-based evaluators. Our approach leverages a multi-LRM jury to generate high-quality synthetic training data via practical data curation techniques and aggregation of evaluation responses across a diverse panel of models. Our results demonstrate that SLMs trained via TQLite achieve strong MQM evaluation performance that far exceeds off-the-shelf evaluation capabilities of standard SLMs, offering a scalable and cost-effective alternative to LLM- and LRM-based evaluators.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.02975 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:05:52.973Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).