Metraipolis › archive › dc_66cefa9db827

StalePO: Anchored Token-Level Preference Optimization using Legacy Post-Edits in Machine Translation — arXiv:2609.16340

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.16340
License (as archived)
CC0-1.0
Archived
2026-10-15T19:19:15.534Z
SHA-256 of the archived text
b3c6793ca5355830bcbea303ab2225349f7e6814b4155545191186818c98a0b4
Stable URL of this page
https://metraipolis.com/documento/dc_66cefa9db827

How to cite

StalePO: Anchored Token-Level Preference Optimization using Legacy Post-Edits in Machine Translation — arXiv:2609.16340 — arXiv (en), https://arxiv.org/abs/2609.16340 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_66cefa9db827 (Metraipolis archive dc_66cefa9db827, sha256 b3c6793ca5355830bcbea303ab2225349f7e6814b4155545191186818c98a0b4)
BibTeX
@misc{dc66cefa9db827,
  title = {StalePO: Anchored Token-Level Preference Optimization using Legacy Post-Edits in Machine Translation — arXiv:2609.16340},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.16340},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_66cefa9db827; sha256 b3c6793ca5355830bcbea303ab2225349f7e6814b4155545191186818c98a0b4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16340. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Machine translation systems are periodically upgraded to stronger models, but the available preference signal is human post-edits of an older system's outputs, which the newer model may already surpass. Moreover, collecting fresh post-edits for every new model is prohibitively expensive. We call this the Stale Preference problem. Standard DPO can fail in this setting: it may increase the likelihood of inferior post-edits, erode the model's existing quality, and fail to provide the per-token control needed to correct localized errors. We introduce StalePO, an objective derived from three requirements this regime imposes. Likelihood movement must be downward on both responses, the policy must be anchored to its own base response, and the KL constraint must apply at the token level. These requirements are jointly necessary. In ablations, each mechanism in isolation leaves the model's performance indistinguishable from the base model, and only their combination converts stale feedback into gains. On English-to-Hindi and English-to-Turkish localization data, StalePO improves the fraction of segments passing all LLM-as-judge MQM quality checks by 14.9 and 4.6 percentage points, respectively, with gains concentrated on style and fluency. A human evaluation under the same framework confirms these gains on English-to-Hindi, raising the fraction of segments passing all seven human checks by 13.8 percentage points.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.16340 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:32:32.803Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).