Metraipolis › archive › dc_4e84ec4ecf0c

KLEAR: Knowledge-Matched LLM Evaluation in Korean across Application Requirements via Controlled Benchmark Transformation — arXiv:2611.09223

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09223
License (as archived)
CC0-1.0
Archived
2026-11-09T22:35:58.144Z
SHA-256 of the archived text
74cfd3923c4ba40f04716b2c7b006fb1f4d85176678de983e917244c07c68d96
Stable URL of this page
https://metraipolis.com/documento/dc_4e84ec4ecf0c

How to cite

KLEAR: Knowledge-Matched LLM Evaluation in Korean across Application Requirements via Controlled Benchmark Transformation — arXiv:2611.09223 — arXiv (en), https://arxiv.org/abs/2611.09223 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e84ec4ecf0c (Metraipolis archive dc_4e84ec4ecf0c, sha256 74cfd3923c4ba40f04716b2c7b006fb1f4d85176678de983e917244c07c68d96)
BibTeX
@misc{dc4e84ec4ecf0c,
  title = {KLEAR: Knowledge-Matched LLM Evaluation in Korean across Application Requirements via Controlled Benchmark Transformation — arXiv:2611.09223},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09223},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e84ec4ecf0c; sha256 74cfd3923c4ba40f04716b2c7b006fb1f4d85176678de983e917244c07c68d96},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09223. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing Korean benchmarks for Large Language Models (LLMs) evaluate performance across diverse knowledge domains, but their accuracy scores cannot distinguish whether incorrect responses arise from a lack of required knowledge or a failure to apply that knowledge correctly. Our analysis of existing Korean benchmarks reveals that most questions can be answered through factual recall alone, while those demanding greater application also differ in required knowledge, making it difficult to isolate application ability. To address this, we introduce KLEAR, a framework that constructs knowledge-matched contrasts by varying application demand while preserving the required knowledge. During construction, KLEAR identifies the knowledge underlying each source question as an explicit anchor bundle to guide generation of questions requiring more complex application. Generated candidates then undergo automated and human validation, with iterative revision based on evaluation feedback. Experiments demonstrate that the resulting knowledge-matched sets largely satisfy the intended constraints, enabling controlled evaluation across diverse LLMs. These evaluations reveal that increasing application demand exposes failures that persist even when the required knowledge is provided.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.09223 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:31:01.947Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).