Metraipolis › archive › dc_fe3df6d32ef9
Verifiable Social Reasoning for LLM Assistants — arXiv:2609.17496
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17496
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T18:32:29.943Z
- SHA-256 of the archived text
25574f86288e07a3c37c721f26d0e00ab06e5d75964d4f068a9b53a5728e06db- Stable URL of this page
https://metraipolis.com/documento/dc_fe3df6d32ef9
How to cite
Verifiable Social Reasoning for LLM Assistants — arXiv:2609.17496 — arXiv (en), https://arxiv.org/abs/2609.17496 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fe3df6d32ef9 (Metraipolis archive dc_fe3df6d32ef9, sha256 25574f86288e07a3c37c721f26d0e00ab06e5d75964d4f068a9b53a5728e06db)
BibTeX
@misc{dcfe3df6d32ef9,
title = {Verifiable Social Reasoning for LLM Assistants — arXiv:2609.17496},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17496},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fe3df6d32ef9; sha256 25574f86288e07a3c37c721f26d0e00ab06e5d75964d4f068a9b53a5728e06db},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17496. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM assistants are widely used for daily social advice, yet evaluating their social reasoning in such consultation settings remains challenging since (i) it requires setups where the assistant learns about social situations from subjective user narratives, and (ii) social properties, such as others' intentions, typically lack verifiable ground truth. To address these challenges, we introduce Fuse, a multi-agent simulation framework for studying user-mediated social reasoning. In Fuse, a target agent with a hidden motive interacts with other agents including one representing the user, who then consults the evaluated assistant to infer the target's motive, providing verifiable ground truth by construction. Simulation faithfulness is validated through a human study with 24k annotations. We apply Fuse to 12 LLMs and demonstrate its analytical utility by systematically isolating key factors, showing that (i) user mediation compounds the inherent difficulty of social reasoning; (ii) LLMs exhibit systematic sensitivity to biased user framing; (iii) models can require more details than humans need to reach a correct prediction; and (iv) longer conversations do not always improve performance despite providing opportunities for clarifying questions. We open-source Fuse and a dataset with 21k examples. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.17496 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T18:03:09.543Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).