Metraipolis › archive › dc_cb698bf829db

The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests — arXiv:2608.15188

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.15188
License (as archived): CC0-1.0
Archived: 2026-10-18T18:02:08.883Z
SHA-256 of the archived text: e42454af781480afd349e127ff4107ee5add8946318566c32cd52e18a9735845
Stable URL of this page: https://metraipolis.com/documento/dc_cb698bf829db

How to cite

The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests — arXiv:2608.15188 — arXiv (en), https://arxiv.org/abs/2608.15188 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cb698bf829db (Metraipolis archive dc_cb698bf829db, sha256 e42454af781480afd349e127ff4107ee5add8946318566c32cd52e18a9735845)

BibTeX

@misc{dccb698bf829db,
  title = {The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests — arXiv:2608.15188},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15188},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cb698bf829db; sha256 e42454af781480afd349e127ff4107ee5add8946318566c32cd52e18a9735845},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15188. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We evaluate the quality of Claude AI-written Python tests against human-written Python tests from two established open-source projects Django and Pandas. Hundreds of tests per corpus are scored under one identical protocol. Using one-sided non-inferiority bounds, we find that the tests written by recent Claude models (Sonnet/Opus 4.6 and later) are no weaker than the two human-written corpora. In this study: (i) the AI-written corpus is tests from real tools, not synthetic tests generated in isolation against a fixed target, the setup used by every other AI-test-generation study we are aware of; (ii) every test is individually scored under three independent fault-injection protocols plus a seven-axis qualitative design rubric, allowing methods to cross-validate each other; (iii) tests are scored individually, rather than suite-level, identifying exactly which specific tests need attention.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.15188 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.995Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).