Metraipolis › archive › dc_7a1a61326e98

A Case Study in Assuring AI-Written Software — arXiv:2610.08651

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.08651
License (as archived)
CC0-1.0
Archived
2026-10-07T11:17:27.012Z
SHA-256 of the archived text
01c289a4418a7d2418c908b47258741b99b55dae7787c8b727f9b0f98f81922f
Stable URL of this page
https://metraipolis.com/documento/dc_7a1a61326e98

How to cite

A Case Study in Assuring AI-Written Software — arXiv:2610.08651 — arXiv (en), https://arxiv.org/abs/2610.08651 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a1a61326e98 (Metraipolis archive dc_7a1a61326e98, sha256 01c289a4418a7d2418c908b47258741b99b55dae7787c8b727f9b0f98f81922f)
BibTeX
@misc{dc7a1a61326e98,
  title = {A Case Study in Assuring AI-Written Software — arXiv:2610.08651},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08651},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a1a61326e98; sha256 01c289a4418a7d2418c908b47258741b99b55dae7787c8b727f9b0f98f81922f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08651. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Software-engineering agents can enable people without formal software training to build systems they could not otherwise implement and simultaneously can produce more code than even experts can meaningfully inspect. In both cases, exhaustive code review is not reliable as the sole basis for human control. We report a case study of a production healthcare platform built through coding agents and governed by an operator without formal software-engineering training. Over time, its workflow grew into a human-led meta-agent system where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The operator found that tests, monitors and reviewing agents used to supervise the system were fallible. Some monitors measured proxies rather than outcomes, some audits failed silently, missing checks disappeared from reported results and one automated repair caused operational disruption. In this case, human control depended on keeping the intended outcome, the evidence used to judge it, the agents' permissions and the final human decision were all tied to the same underlying objective.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2610.08651 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:09.489Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).