Metraipolis › archive › dc_aa69ed39352b
Can We Triage LLM Translation Errors in Classical Texts Without Human References? Source Novelty, GEMBA Scoring, and Budgeted Review through Pali-to-English Translation — arXiv:2609.14963
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.14963
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:15:37.805Z
- SHA-256 of the archived text
0501f3e5a1de3c7eb6a2c9d50a072e57ef69b0be587935892c5b623d0da494bf- Stable URL of this page
https://metraipolis.com/documento/dc_aa69ed39352b
How to cite
Can We Triage LLM Translation Errors in Classical Texts Without Human References? Source Novelty, GEMBA Scoring, and Budgeted Review through Pali-to-English Translation — arXiv:2609.14963 — arXiv (en), https://arxiv.org/abs/2609.14963 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa69ed39352b (Metraipolis archive dc_aa69ed39352b, sha256 0501f3e5a1de3c7eb6a2c9d50a072e57ef69b0be587935892c5b623d0da494bf)
BibTeX
@misc{dcaa69ed39352b,
title = {Can We Triage LLM Translation Errors in Classical Texts Without Human References? Source Novelty, GEMBA Scoring, and Budgeted Review through Pali-to-English Translation — arXiv:2609.14963},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.14963},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa69ed39352b; sha256 0501f3e5a1de3c7eb6a2c9d50a072e57ef69b0be587935892c5b623d0da494bf},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14963. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
As large language models become capable translators of classical texts, a key challenge is deciding which outputs need expert review when no human reference exists. This study tests reference-free error triage through Pali-to-English translation. Three LLMs translated 15,493 passages. Five signals were compared: source novelty, source-candidate embedding distance, peer-translation disagreement, English-to-Pali backtranslation, and no-reference GEMBA scoring. Signals were calibrated on a 3,000-item reference-informed LLM-adjudicated sample and checked against a 500-item author-adjudicated anchor. Human references supported calibration and validation only; they were never used to compute the risk signals. Source novelty was a useful source-side risk prior but not a per-candidate error detector. Peer disagreement and backtranslation provided secondary signal. The strongest method was no-reference GEMBA scoring by a panel of models generally regarded as stronger than the translators: reviewing the top 10% by GEMBA risk captured 81.6% of panel-major errors in the calibration set. GEMBA also remained the best reference-free signal against the author anchor. A same-tier panel, with self-scoring excluded, remained useful but performed worse, indicating that evaluator strength matters beyond the prompt alone. A budgeted workflow is proposed, combining source novelty, peer disagreement, and stronger candidate-aware judging to allocate human review. Transfer to other classical languages, including Latin, Ancient Greek, and Sanskrit, remains to be tested. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.14963 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:51:09.915Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).