Metraipolis › archive › dc_e69d45ea0a28

Evaluating Multilingual Sentence Embeddings for Translation Error Detection:An English--Greek Contrastive Study — arXiv:2608.28776

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.28776
License (as archived)
CC0-1.0
Archived
2026-10-17T08:06:47.183Z
SHA-256 of the archived text
8a95f41577b380e9d96760c44e55c5a17e067b929e6d03444aa5fa9691db1085
Stable URL of this page
https://metraipolis.com/documento/dc_e69d45ea0a28

How to cite

Evaluating Multilingual Sentence Embeddings for Translation Error Detection:An English--Greek Contrastive Study — arXiv:2608.28776 — arXiv (en), https://arxiv.org/abs/2608.28776 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e69d45ea0a28 (Metraipolis archive dc_e69d45ea0a28, sha256 8a95f41577b380e9d96760c44e55c5a17e067b929e6d03444aa5fa9691db1085)
BibTeX
@misc{dce69d45ea0a28,
  title = {Evaluating Multilingual Sentence Embeddings for Translation Error Detection:An English--Greek Contrastive Study — arXiv:2608.28776},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.28776},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e69d45ea0a28; sha256 8a95f41577b380e9d96760c44e55c5a17e067b929e6d03444aa5fa9691db1085},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28776. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multilingual sentence embeddings are increasingly used to estimate semantic similarity across languages, yet their sensitivity to fine-grained translation errors remains insufficiently understood. This study investigates whether general-purpose multilingual embedding models can distinguish correct English-Greek translations from minimally modified erroneous alternatives. A contrastive dataset was developed from FLORES+ sentence-aligned reference translations and reviewed by two translation experts. It contains 1,850 examples across ten core and five exploratory error categories, covering factual, lexical-semantic, grammatical, relational, referential, and discourse-level phenomena. Five multilingual sentence-embedding models (BGE-M3, Multilingual E5, Multilingual MPNet, LaBSE, and Jina Embeddings v3) were evaluated using cosine similarity between each English source sentence and its correct and erroneous Greek translations. A reference-free COMETKiwi model was also evaluated as an MT quality-estimation baseline. Performance was assessed through contrastive accuracy and score margins for category-specific sensitivity. BGE-M3 achieved the highest accuracy among embedding models at 89.30 percent, while COMETKiwi achieved 94.49 percent. Embedding models detected explicit factual and lexical changes more reliably than tense-and-aspect and pronoun-coreference errors. COMETKiwi improved performance on several difficult categories, including tense and aspect, pronoun and coreference, and semantic-role errors, but showed lower sensitivity to date-and-time errors and underperformed the embedding models on numbers. The results show complementary error-sensitivity profiles: multilingual sentence embeddings provide useful semantic adequacy signals but are better suited as components of broader translation-evaluation frameworks than as standalone metrics.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.28776 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:47:52.657Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).