Metraipolis › archive › dc_4ba2a4ca37bb

Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations? — arXiv:2608.08283

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08283
License (as archived)
CC0-1.0
Archived
2026-10-19T08:49:57.287Z
SHA-256 of the archived text
30a37b650d5633e9f0ab7d79ebaff289fe0a79efc6562a8df0f33fcf67b41ac5
Stable URL of this page
https://metraipolis.com/documento/dc_4ba2a4ca37bb

How to cite

Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations? — arXiv:2608.08283 — arXiv (en), https://arxiv.org/abs/2608.08283 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4ba2a4ca37bb (Metraipolis archive dc_4ba2a4ca37bb, sha256 30a37b650d5633e9f0ab7d79ebaff289fe0a79efc6562a8df0f33fcf67b41ac5)
BibTeX
@misc{dc4ba2a4ca37bb,
  title = {Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations? — arXiv:2608.08283},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08283},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4ba2a4ca37bb; sha256 30a37b650d5633e9f0ab7d79ebaff289fe0a79efc6562a8df0f33fcf67b41ac5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08283. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Although large language models can translate some historical languages surprisingly well, their usefulness in digital humanities workflows is limited by the lack of reliable evaluation. We investigate whether existing automatic evaluation metrics developed for modern languages are reliable in this setting, using translation from Classical Chinese to English as a test case. We introduce a diagnostic framework based on minimal pairs capturing error types salient in scholarly use, probing both reference-based and reference-free metrics for error sensitivity and tolerance to valid variation. We find that all metrics exhibit blind spots, however MetricX24 performs best overall. Our findings highlight the need for more robust and interpretable metrics for historically and culturally distinct translation settings.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.08283 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.855Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).