Metraipolis › archive › dc_201aaf22892c

ChemCLIR-Bench: Benchmarking Cross-Lingual Information Retrieval in Multilingual Chemical Patents — arXiv:2609.23231

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.23231
License (as archived)
CC0-1.0
Archived
2026-10-09T06:49:46.979Z
SHA-256 of the archived text
e68143e9431744fa4c5946b8c95446087d46e602ae8cbe295f65b47411dcea98
Stable URL of this page
https://metraipolis.com/documento/dc_201aaf22892c

How to cite

ChemCLIR-Bench: Benchmarking Cross-Lingual Information Retrieval in Multilingual Chemical Patents — arXiv:2609.23231 — arXiv (en), https://arxiv.org/abs/2609.23231 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_201aaf22892c (Metraipolis archive dc_201aaf22892c, sha256 e68143e9431744fa4c5946b8c95446087d46e602ae8cbe295f65b47411dcea98)
BibTeX
@misc{dc201aaf22892c,
  title = {ChemCLIR-Bench: Benchmarking Cross-Lingual Information Retrieval in Multilingual Chemical Patents — arXiv:2609.23231},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.23231},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_201aaf22892c; sha256 e68143e9431744fa4c5946b8c95446087d46e602ae8cbe295f65b47411dcea98},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23231. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Cross-lingual information retrieval (CLIR) is increasingly important in multi-national industries, where critical technical evidence may exist in a different language than the query. However, existing benchmarks do not adequately capture domain-specific cross-lingual retrieval or the retrieval-depth and recoverability failures that aggregate recall hides. In this work, we benchmark CLIR in the chemical domain, with a focus on patent data. We construct a multilingual dataset from Google Patents and the European Patent Office (EPO) data, spanning five languages (covering major Eastern and Western languages) and reflecting the diversity and complexity of real-world industrial documentation. Using this dataset, we systematically evaluate eight state-of-the-art embedding models for cross-lingual retrieval. Our results show a substantial performance gap between monolingual and cross-lingual settings: for the best-performing model, Recall@10 drops from 0.72 to 0.53 in cross-lingual setting. Retrieval depth also degrades significantly, with relevant documents ranked lower across languages in cross-lingual scenarios. Furthermore, some multilingual embedding models that perform strongly in monolingual settings exhibit sharp declines when queries and documents are in different languages, providing practical insights for model selection in cross-lingual use cases. These findings highlight critical limitations of current approaches and emphasize the need for more robust cross-lingual retrieval methods in domain-specific settings. Our benchmark provides actionable insights for model selection and establishes a controlled diagnostic evaluation framework for CLIR over industrial technical text. Data and code are publicly available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.23231 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:44.982Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).