Metraipolis › archive › dc_691693bb428c
A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents — arXiv:2609.17698
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17698
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T15:38:48.178Z
- SHA-256 of the archived text
23cfb9333b7d06fab154fa3a978ed4faed0d8a7dece6a80519cd2c7e735caf92- Stable URL of this page
https://metraipolis.com/documento/dc_691693bb428c
How to cite
A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents — arXiv:2609.17698 — arXiv (en), https://arxiv.org/abs/2609.17698 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_691693bb428c (Metraipolis archive dc_691693bb428c, sha256 23cfb9333b7d06fab154fa3a978ed4faed0d8a7dece6a80519cd2c7e735caf92)
BibTeX
@misc{dc691693bb428c,
title = {A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents — arXiv:2609.17698},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17698},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_691693bb428c; sha256 23cfb9333b7d06fab154fa3a978ed4faed0d8a7dece6a80519cd2c7e735caf92},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17698. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM)-based agents are increasingly used across software engineering, web automation, research, and productivity applications. Their integration of planning, memory, tool use, code execution, and external interactions enables greater autonomy but also introduces new reliability, safety, and security risks. We present a large-scale empirical study of quality assurance (QA) practices in 157 open-source LLM-based agent projects with at least 100 GitHub stars. We analyze documentation, source code, configurations, and tests to characterize QA practices across execution surfaces, safeguards, testing artifacts, risk scenarios, and recurring gaps. We find that current QA primarily focuses on basic functionality and high-risk actions, while coverage remains fragmented. Safeguards are inconsistently applied across equivalent execution routes, tests rarely examine boundary, adversarial, or multi-step tool-use failures, and identified risks are seldom translated into end-to-end QA checks. These findings highlight the need to move beyond feature-level testing toward systematic end-to-end validation that ensures agent workflows remain within intended boundaries when interacting with untrusted inputs, tools, persistent state, and external APIs. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-17 in cs.SE). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.17698 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T15:04:30.917Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).