Metraipolis › archive › dc_32aa4e86dc35

Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation — arXiv:2609.14795

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.14795
License (as archived): CC0-1.0
Archived: 2026-10-15T22:53:20.802Z
SHA-256 of the archived text: c88f5d894cd317ae6c8b0e2601f99fd944f79856169be4248ef18ca3e95fee90
Stable URL of this page: https://metraipolis.com/documento/dc_32aa4e86dc35

How to cite

Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation — arXiv:2609.14795 — arXiv (en), https://arxiv.org/abs/2609.14795 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_32aa4e86dc35 (Metraipolis archive dc_32aa4e86dc35, sha256 c88f5d894cd317ae6c8b0e2601f99fd944f79856169be4248ef18ca3e95fee90)

BibTeX

@misc{dc32aa4e86dc35,
  title = {Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation — arXiv:2609.14795},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.14795},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_32aa4e86dc35; sha256 c88f5d894cd317ae6c8b0e2601f99fd944f79856169be4248ef18ca3e95fee90},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14795. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

There is a tradeoff in machine translation meta-evaluation between prioritizing alignment with adequacy versus fluency. The balance depends on the combination of translation systems in the meta-evaluation dataset. This system set is a small, filtered sample whose characteristics change heavily across years and language pairs; it does not represent the true system distribution. Consequently, the adequacy-fluency balance is often unrepresentative and subject to change. For sensitive domains, controlling this balance is critical. We expose this balance as a tunable choice. To achieve a target balance, we reweight existing systems while minimizing distortion from uniform weighting, ensuring the evaluated systems remain real and representative. We provide an exact optimization algorithm with theoretical guarantees and pruning mechanisms to compute these weights. To validate meta-evaluation internal consistency, we design a scorer-augmentation framework that establishes a known relative identity for the scorers. Results demonstrate that our reweighting method effectively controls the adequacy-fluency balance and preserves the internal consistency, outperforming prior approaches. Finally, we analyze the performance of popular scorers across a sweep of this parameter.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.14795 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T22:29:03.732Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).