Metraipolis › archive › dc_cce920f0fabf
LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? — arXiv:2610.04118
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.04118
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:28:01.596Z
- SHA-256 of the archived text
ae50c9e670165842ec167b4ca7d187e558d7e15b0dafc66d6419765f207cbe56- Stable URL of this page
https://metraipolis.com/documento/dc_cce920f0fabf
How to cite
LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? — arXiv:2610.04118 — arXiv, https://arxiv.org/abs/2610.04118 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cce920f0fabf (Metraipolis archive dc_cce920f0fabf, sha256 ae50c9e670165842ec167b4ca7d187e558d7e15b0dafc66d6419765f207cbe56)
BibTeX
@misc{dccce920f0fabf,
title = {LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? — arXiv:2610.04118},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.04118},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cce920f0fabf; sha256 ae50c9e670165842ec167b4ca7d187e558d7e15b0dafc66d6419765f207cbe56},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04118. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Long-context LLMs can now ingest entire online discussion threads, but understanding their social discourse requires more than reading a long document: models must track parent-reply relations, turning points, scoped subtrees, cross-branch contrasts, and participant trajectories. To test this structure-aware social reasoning, we introduce LongSocialBench, a benchmark of 1,462 verified human-authored multiple-choice items drawn from 94 complete Hacker News, Stack Exchange, and Reddit r/ChangeMyView episodes, with a median length of approximately 73K tokens. Each item pairs a complete serialized discussion and reply structure with a four-option question, requiring models to recover structured social evidence. Released items are verified for answerability, option uniqueness, and evidence grounding. Across 18 models and 29 evaluation settings, current long-context workflows remain far below human performance. The best individual result comes from Claude-Opus-4.7, which reaches 63.0% when prompted to eliminate incorrect options before answering, compared with 72.4% for independent human readers. Averaged across all 18 models, the full-context Baseline scores 43.9%. Supplying the gold evidence scope raises this to 55.0%, showing that substantial errors remain even after the relevant thread region is identified. LongSocialBench shows that the missing capability is not context access or prompting alone, but social understanding over structured reply trees. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.04118 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:20.250Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).