Metraipolis › archive › dc_85f2e30ccd47

Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry — arXiv:2609.19154

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.19154
License (as archived)
CC0-1.0
Archived
2026-10-10T03:47:56.915Z
SHA-256 of the archived text
3c94aab53450345321e5094fd904db953ad09e91fc922500a05d3be872cf389e
Stable URL of this page
https://metraipolis.com/documento/dc_85f2e30ccd47

How to cite

Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry — arXiv:2609.19154 — arXiv (en), https://arxiv.org/abs/2609.19154 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_85f2e30ccd47 (Metraipolis archive dc_85f2e30ccd47, sha256 3c94aab53450345321e5094fd904db953ad09e91fc922500a05d3be872cf389e)
BibTeX
@misc{dc85f2e30ccd47,
  title = {Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry — arXiv:2609.19154},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19154},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_85f2e30ccd47; sha256 3c94aab53450345321e5094fd904db953ad09e91fc922500a05d3be872cf389e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19154. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While Large Language Models (LLMs) achieve high accuracy on established Classical Chinese Poetry benchmarks, it remains challenging to distinguish transferable Linguistic-Aesthetic Reasoning from reliance on familiar pre-training patterns. To address this issue, we introduce Neo-Classic, an evaluation benchmark that combines a constructionist Out-of-Sample (OOS) dataset with a suite of reverse understanding probes. Unlike traditional benchmarks that rely on verification or generation over historical corpora, Neo-Classic comprises strictly metrical poetry authored by contemporary experts, reducing the possibility of direct retrieval. We evaluate state-of-the-art models, including Qwen3-Max, Gemini-3-Pro, and DeepSeek-V3.2, across five behavioral probes designed to test hierarchical constraint satisfaction. Our results reveal two primary limitations. First, a performance gap of 20 to 50 percent emerges when models transition from historical to contemporary texts. Second, models exhibit substantial difficulties in discourse-level ordering tasks, with standard accuracy remaining low (0 to 13 percent). Although expert-level guidance improves the performance of reasoning-enhanced models to 36 percent, a notable gap with human experts persists. These findings suggest that while current LLMs capture local formal patterns, they struggle with global hierarchical planning required for robust Linguistic-Aesthetic Reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.19154 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:24:08.110Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).