Metraipolis › archive › dc_71ce031e688c
What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports — arXiv:2610.20003
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.20003
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T20:02:16.358Z
- SHA-256 of the archived text
ed19963ccba453d278a7fe5cedb834d27a9aa3bf6bcdf29cd707c91c4a195815- Stable URL of this page
https://metraipolis.com/documento/dc_71ce031e688c
How to cite
What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports — arXiv:2610.20003 — arXiv (en), https://arxiv.org/abs/2610.20003 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_71ce031e688c (Metraipolis archive dc_71ce031e688c, sha256 ed19963ccba453d278a7fe5cedb834d27a9aa3bf6bcdf29cd707c91c4a195815)
BibTeX
@misc{dc71ce031e688c,
title = {What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports — arXiv:2610.20003},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.20003},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_71ce031e688c; sha256 ed19963ccba453d278a7fe5cedb834d27a9aa3bf6bcdf29cd707c91c4a195815},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20003. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Human supervisors rely on coding agents' reports to assess progress, verify repairs, and decide when to intervene. But a supervisor cannot investigate a failure they were never told occurred. Failed attempts, workarounds, and unresolved risks may disappear from routine reports, leaving supervisors unaware of issues that warrant scrutiny. We conduct CTXGAP, a measurement study quantifying the gap in task-state recoverability between agents' accumulated user-facing reports and their full recorded execution contexts. Across 37 coding tasks and 4,799 valid probes, agents answer 93.6% of task-state questions on demand; full-context readers recover 92.7%, while report-only readers recover 52.0%, a 40.7-point gap. At the final checkpoints of 30 verified fixes, the gap remains 34.8 points. Human annotation identifies states mentioned without enough supporting detail as the largest category of unrecovered information, and model-assessed supervision-relevant states retain substantial gaps. Full terminal scrollback improves recovery to 73.6% but still trails full context. Under matched report lengths, context-informed rewrites improve recovery by about eight points, without a detectable targeted-over-natural advantage. Positive gaps persist across harnesses and independent DeepSeek execution and measurement. CTXGAP establishes report-based task-state recoverability as a complementary evaluation dimension. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.20003 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T19:35:48.283Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).