Metraipolis › archive › dc_32ef58ae530d
ReFract: Benchmarking Perspective Awareness in Language Model Agents with Text World Models — arXiv:2610.03356
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.03356
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:49:37.164Z
- SHA-256 of the archived text
d63023e75bc535f945b1088e491ca8e77af47cf47590bc46870bfa461619c0d9- Stable URL of this page
https://metraipolis.com/documento/dc_32ef58ae530d
How to cite
ReFract: Benchmarking Perspective Awareness in Language Model Agents with Text World Models — arXiv:2610.03356 — arXiv, https://arxiv.org/abs/2610.03356 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_32ef58ae530d (Metraipolis archive dc_32ef58ae530d, sha256 d63023e75bc535f945b1088e491ca8e77af47cf47590bc46870bfa461619c0d9)
BibTeX
@misc{dc32ef58ae530d,
title = {ReFract: Benchmarking Perspective Awareness in Language Model Agents with Text World Models — arXiv:2610.03356},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.03356},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_32ef58ae530d; sha256 d63023e75bc535f945b1088e491ca8e77af47cf47590bc46870bfa461619c0d9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03356. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Model (LLM) agents are increasingly deployed in high-stakes settings such as industrial maintenance and equipment fault troubleshooting, where workers occupy a variety of roles. A capable agent must therefore act in a way that is calibrated to user's role: taking actions and providing information that respect the role's knowledge and capability boundaries. Unlike coding, where mistakes are usually recoverable, agent responses in these settings are enacted on physical equipment, and can therefore cause irreversible equipment damage, production loss, or personnel harm. Existing benchmarks, however, largely overlook the need for agents to infer what a role intends and acting only through tools that role may legitimately use, a capability which we term Perspective Awareness. To this end, we introduce ReFract, a benchmark of 150 expert-validated entries in which an agent must act differently in response to the same query depending on user's role. Entries of ReFract are grounded in anonymized queries from domain support conversations, against which we construct Text World Models that simulate the agent's operating environments and assemble perspective-aware action trajectories. State-of-the-art LLMs solve at most 69% of the tasks with more than 50% of their trajectories contain attempts of taking perspective-violating actions. ReFract exposes perspective awareness as a distinct, largely unsolved axis of agent evaluation and motivates agents that calibrate not just how to act, but for whom. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.03356 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.692Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).