Metraipolis › archive › dc_ba53360ecf1c

Do Reasoning Representations Help Humans Evaluate LLM Outputs? — arXiv:2609.09038

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.09038
License (as archived)
CC0-1.0
Archived
2026-10-16T13:46:56.174Z
SHA-256 of the archived text
2db381249f610d8ed253016fb8ad1e72ba161bedd0731939c6addfc6019fabb1
Stable URL of this page
https://metraipolis.com/documento/dc_ba53360ecf1c

How to cite

Do Reasoning Representations Help Humans Evaluate LLM Outputs? — arXiv:2609.09038 — arXiv (en), https://arxiv.org/abs/2609.09038 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ba53360ecf1c (Metraipolis archive dc_ba53360ecf1c, sha256 2db381249f610d8ed253016fb8ad1e72ba161bedd0731939c6addfc6019fabb1)
BibTeX
@misc{dcba53360ecf1c,
  title = {Do Reasoning Representations Help Humans Evaluate LLM Outputs? — arXiv:2609.09038},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09038},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ba53360ecf1c; sha256 2db381249f610d8ed253016fb8ad1e72ba161bedd0731939c6addfc6019fabb1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09038. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reasoning representations are increasingly used as explanations for large language model outputs. Yet they are typically evaluated with model-centric criteria, such as answer accuracy and faithfulness, leaving it unclear whether they help people evaluate model responses. In this work, we study reasoning representations as human-facing interfaces rather than proxies for model reasoning ability. We conduct a controlled human study of six reasoning formats across tasks of varying complexity, supported by a web-based framework that randomizes task domains, problem instances, and representation order. The study collects fine-grained judgments of structural understanding, error detection and localization, and trust calibration. Our study shows a mismatch between perceived preference and support for human evaluation. Participants prefer planning- and decomposition-based representations, but simpler chain-of-thought traces better support verification, trust, and interpretability. Preferred representations also introduce calibration risks, with more false alarms on correct traces and high trust despite low willingness to verify.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.HC).
Categorie arXiv: Machine Learning (cs.LG) ; Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2609.09038 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:28:43.177Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).