Metraipolis › archive › dc_c7514afa5ca7
L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages — arXiv:2608.15535
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.15535
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T18:58:44.944Z
- SHA-256 of the archived text
c1fad2f3462274b7c826151cfc5a4eb1d84a36b33636bec90cf58967b762c56e- Stable URL of this page
https://metraipolis.com/documento/dc_c7514afa5ca7
How to cite
L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages — arXiv:2608.15535 — arXiv (en), https://arxiv.org/abs/2608.15535 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c7514afa5ca7 (Metraipolis archive dc_c7514afa5ca7, sha256 c1fad2f3462274b7c826151cfc5a4eb1d84a36b33636bec90cf58967b762c56e)
BibTeX
@misc{dcc7514afa5ca7,
title = {L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages — arXiv:2608.15535},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.15535},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c7514afa5ca7; sha256 c1fad2f3462274b7c826151cfc5a4eb1d84a36b33636bec90cf58967b762c56e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15535. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs). The benchmark comprises 3,471 curriculum-grounded English question--answer pairs spanning nine domains, curated from educational curricula, competitive examination materials, and domain-specific reference books. We introduce a practical hybrid construction strategy that combines context-grounded LLM-based question generation and validation with semantic deduplication and human verification, enabling scalable creation of benchmark data while preserving annotation quality. The benchmark is translated into 19 Indic languages, yielding a publicly released multilingual dataset of 69,420 question--answer pairs across 20 languages. We evaluate six LLMs under three protocols: LLM-as-a-judge and two deterministic lexical criteria, exact-substring and word-overlap matching. All three produce almost the same model ranking, showing that the results do not depend on the choice of judge. The frontier commercial model leads by a wide margin, and among open-weight models Gemma4 31B outperforms the Indic-specialised Sarvam 30B in every evaluated Indic language. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.15535 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T18:06:57.857Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).