Metraipolis › archive › dc_75aee2c0698f
Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in Translation — arXiv:2611.09785
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09785
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:36:02.525Z
- SHA-256 of the archived text
0d849a45bfba4ef04453001a75480ff8d3753c9a6763063d3220c9c38963fa2f- Stable URL of this page
https://metraipolis.com/documento/dc_75aee2c0698f
How to cite
Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in Translation — arXiv:2611.09785 — arXiv (en), https://arxiv.org/abs/2611.09785 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_75aee2c0698f (Metraipolis archive dc_75aee2c0698f, sha256 0d849a45bfba4ef04453001a75480ff8d3753c9a6763063d3220c9c38963fa2f)
BibTeX
@misc{dc75aee2c0698f,
title = {Are These Programmers Ripe Enough to Eat? Evaluating How LLMs Handle Lexical Knowledge Conflicts in Translation — arXiv:2611.09785},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09785},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_75aee2c0698f; sha256 0d849a45bfba4ef04453001a75480ff8d3753c9a6763063d3220c9c38963fa2f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09785. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
External lexical resources can help large language models (LLMs) translate texts in low-resource languages and specialized domains, but these resources may conflict with models' prior lexical knowledge. To isolate this conflict, we construct a controlled Mandarin-to-English translation task in which familiar source words are assigned new meanings within and across semantic categories. We treat these prompt-supplied meanings as contextual knowledge (CK) and systematically evaluate the effects of word frequency, sentence context, and prompt framing on the translated outputs. Across nine models, CK reliance is greater for less frequent source words and in sentence contexts that support the supplied meanings. Prompt framing can reduce CK reliance, but seven of nine models favor CK even when explicitly instructed to ignore the supplied meanings. Within model families, larger models are more likely than smaller models to utilize CK in their outputs. Three follow-up experiments further show that modal weakening lowers CK reliance, the effect of repetition depends on prompt layout, and the effect of reasoning varies across models. Overall, our controlled study reveals how CK reliance varies across conditions and models, motivating new methods to detect and evaluate lexical conflicts and guide models' lexical choices in LLM-based translation. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.09785 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:31:01.950Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).