Metraipolis › archive › dc_ddfb53fc1269

KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms — arXiv:2609.19916

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.19916
License (as archived): CC0-1.0
Archived: 2026-10-10T03:23:53.409Z
SHA-256 of the archived text: 34f800676e1b4c03035dea83843b36699ca91238e56b4237f638f46fb205207e
Stable URL of this page: https://metraipolis.com/documento/dc_ddfb53fc1269

How to cite

KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms — arXiv:2609.19916 — arXiv (en), https://arxiv.org/abs/2609.19916 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ddfb53fc1269 (Metraipolis archive dc_ddfb53fc1269, sha256 34f800676e1b4c03035dea83843b36699ca91238e56b4237f638f46fb205207e)

BibTeX

@misc{dcddfb53fc1269,
  title = {KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms — arXiv:2609.19916},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19916},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ddfb53fc1269; sha256 34f800676e1b4c03035dea83843b36699ca91238e56b4237f638f46fb205207e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19916. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are typically evaluated on static benchmarks, even though natural language constantly evolves through newly emerging words and meanings. Existing Korean benchmarks are centered on established vocabulary and therefore provide limited coverage of such recent lexical change, and their English-oriented design makes it difficult to assess the typological properties of Korean, in which content words combine productively with functional morphemes. In this paper, we introduce KoNeoBench, a benchmark for evaluating LLMs' understanding of Korean neologisms. KoNeoBench is built on 1,785 Korean neologisms attested in online news since 2020 and curated through expert lexicographic review. Each entry provides usage examples, word-formation analyses, and dictionary-style definitions. Based on this resource, we define four tasks and report results on recent models, together with a human baseline. Our experiments show that current LLMs exhibit clear limitations in recovering source components, distinguishing semantic categories, and generating accurate definitions. These results reveal specific aspects of recent Korean lexical change that remain challenging for current LLMs. KoNeoBench is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.19916 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:04:13.874Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).