Metraipolis › archive › dc_54dd3333c535
When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents — arXiv:2611.09802
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09802
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:35:16.444Z
- SHA-256 of the archived text
b9fb9b568341ad9b7546d5d14868c4f7a018b70c5b6636ae6a9f8bde2d0785df- Stable URL of this page
https://metraipolis.com/documento/dc_54dd3333c535
How to cite
When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents — arXiv:2611.09802 — arXiv (en), https://arxiv.org/abs/2611.09802 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_54dd3333c535 (Metraipolis archive dc_54dd3333c535, sha256 b9fb9b568341ad9b7546d5d14868c4f7a018b70c5b6636ae6a9f8bde2d0785df)
BibTeX
@misc{dc54dd3333c535,
title = {When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents — arXiv:2611.09802},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09802},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_54dd3333c535; sha256 b9fb9b568341ad9b7546d5d14868c4f7a018b70c5b6636ae6a9f8bde2d0785df},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09802. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM-based coding agents such as Claude Code are guided by repository instruction files (also called context files or agent configuration files) like this http URL and this http URL , which are read in every agent session. As any documentation artifact, these files can fall behind the code and become stale (e.g., a test folder moves, but the file still points to the old one). In technical debt terms, such a stale instruction is documentation debt, and its interest is the cost it adds to every later agent session. Evaluations that check only task success can miss this cost, and the stale file can persist. In this study, we ask what leads agents to act on a stale instruction, how it affects their behavior and at what cost, and how they recover. We study three real-world stale instructions from active open-source repositories in depth, each a different kind of guidance: a test command, a code location, and a procedure for adding a component. Agents fix 22 maintenance tasks under three versions of each instruction (left stale, corrected to the current path, and removed), with the code and the task held fixed, and we analyze 732 trajectories of two models at the level of individual tool calls. We find that agents follow stale guidance when it is relevant to their tasks, sometimes even when correct information is available elsewhere. On the stale test command, Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command; in the other cases, we could not detect a difference. Although agents generally recovered from the immediate failures, none repaired the underlying instruction file. Correcting the instructions eliminated operations on obsolete paths, while removing them prevented most. Our findings show that task success alone can hide the costs of documentation debt, highlighting the need to evaluate agent behavior beyond final outcomes. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.09802 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:30:47.342Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).