Metraipolis › archive › dc_afdcb1aab667

E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets — arXiv:2609.11334

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11334
License (as archived)
CC0-1.0
Archived
2026-10-16T05:27:02.072Z
SHA-256 of the archived text
6e7f2fa9266794c5bb34d43eccd76abc79432a247b7e93dae0567cbd31c3706d
Stable URL of this page
https://metraipolis.com/documento/dc_afdcb1aab667

How to cite

E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets — arXiv:2609.11334 — arXiv (en), https://arxiv.org/abs/2609.11334 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_afdcb1aab667 (Metraipolis archive dc_afdcb1aab667, sha256 6e7f2fa9266794c5bb34d43eccd76abc79432a247b7e93dae0567cbd31c3706d)
BibTeX
@misc{dcafdcb1aab667,
  title = {E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets — arXiv:2609.11334},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11334},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_afdcb1aab667; sha256 6e7f2fa9266794c5bb34d43eccd76abc79432a247b7e93dae0567cbd31c3706d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11334. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Natural Language Inference processes pairs of sentences to extract their semantic relations. NLI has been a hot research topic, integrated as a main component in other NLP applications. Despite significant advancements in textual inference across various languages all around the world, Arabic language still suffers from limited resources in this domain. To address this gap, this paper introduces E-CONAN benchmarks that are composed of sentences pairs from various sources: (1) automatically-translated pairs, (2) human-validated machine-translated pairs, (3) hand-crafted pairs from teaching Arabic as foreign language books, and (4) headlines pairs from different news channels containing rumors. E-CONAN contains two benchmark datasets, E-CONAN-2, a 2-way dataset (RTE) and E-CONAN-3, a 3-way dataset (NLI). Additionally, we have used E-CONAN benchmarks to evaluate 9 state-of-the-art multilingual pretrained models using zero-shot classification. Models were evaluated across the ArNLI, XNLI, and E-CONAN datasets. Results show that E-CONAN is a potentially valuable resource for evaluating model generalization and even for fine-tuning pre-trained models. Its diverse composition, derived from a combination of sources, offers a broader and more robust assessment compared to XNLI and ArNLI. In addition, we have evaluated 5 LLMs on E-CONAN-3 dataset. Moreover, we incorporated MARBERT as a representative Arabic-specific baseline and conducted performance evaluation comparison to demonstrate how Arabic-specific models scale against cross-lingual and LLM-based approaches on the E-CONAN benchmarks. Furthermore, we conducted detailed qualitative and quantitative error analysis to analyze frequent error patterns. E-CONAN benchmarks will be publicly available, we hope that it will enrich research community in Arabic textual entailment and natural language inference.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.11334 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:17:52.150Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).