Metraipolis › archive › dc_fbc79dafd6bd
You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments — arXiv:2609.04384
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04384
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:26:06.937Z
- SHA-256 of the archived text
97a03014668f4f7dba873e9813293f1a4fbead704eacdd91675a847c6707b6cc- Stable URL of this page
https://metraipolis.com/documento/dc_fbc79dafd6bd
How to cite
You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments — arXiv:2609.04384 — arXiv (en), https://arxiv.org/abs/2609.04384 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fbc79dafd6bd (Metraipolis archive dc_fbc79dafd6bd, sha256 97a03014668f4f7dba873e9813293f1a4fbead704eacdd91675a847c6707b6cc)
BibTeX
@misc{dcfbc79dafd6bd,
title = {You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments — arXiv:2609.04384},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04384},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fbc79dafd6bd; sha256 97a03014668f4f7dba873e9813293f1a4fbead704eacdd91675a847c6707b6cc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04384. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Chinese online comments often convey social meaning through indirect and playful language that is hard to interpret without context. Existing evaluations largely organize items around predefined phenomena or controlled pragmatic categories, leaving open whether models can distinguish plausible readings of what a naturally occurring comment is doing in a particular exchange. We introduce a benchmark for evaluating whether LLMs can recover such situated pragmatic meanings. From more than 200,000 public Chinese social media interaction records, we construct 4,735 human-validated diagnostic items, each pairing a target comment with reconstructed preceding context and plausible misreadings. We evaluate eight LLMs as both question writers and solvers in a cross-writer setting. The task is challenging: the strongest model achieves 81.42% leave-writer-out accuracy. Across all eight models, the mean leave-writer-out accuracy is 68.70% while human accuracy was 90.8%. Case analysis shows that models often recognize broad irony or playfulness while misidentifying the mechanism or interactional move. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC). Pagina del lavoro: https://arxiv.org/abs/2609.04384 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T22:18:11.487Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).