Metraipolis › archive › dc_bc7544a7b4fb
ASSERT: A Measurement Pipeline for GenAI Audits — arXiv:2608.13840
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.13840
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:21:09.496Z
- SHA-256 of the archived text
aa9d0caf68e5b7435e5623667b8049221c7654e47d0cbceca2a371e02906ba50- Stable URL of this page
https://metraipolis.com/documento/dc_bc7544a7b4fb
How to cite
ASSERT: A Measurement Pipeline for GenAI Audits — arXiv:2608.13840 — arXiv (en), https://arxiv.org/abs/2608.13840 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc7544a7b4fb (Metraipolis archive dc_bc7544a7b4fb, sha256 aa9d0caf68e5b7435e5623667b8049221c7654e47d0cbceca2a371e02906ba50)
BibTeX
@misc{dcbc7544a7b4fb,
title = {ASSERT: A Measurement Pipeline for GenAI Audits — arXiv:2608.13840},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.13840},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc7544a7b4fb; sha256 aa9d0caf68e5b7435e5623667b8049221c7654e47d0cbceca2a371e02906ba50},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13840. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Audits of generative AI (GenAI) systems often summarize behavior as a reported rate: how often the audited system complies with policy. Researchers and stakeholders use that rate to compare systems, track regressions, and gate deployment. A reported rate reflects both the system under audit and the measurement choices behind it, so a change in the rate can leave it unclear whether the system or those choices moved. We introduce ASSERT, a specification-driven measurement pipeline for GenAI audits that ties each reported rate to a written specification of the measurement choices used to produce it. ASSERT helps draft a behavioral rubric and test cases, then runs the audit against a GenAI system and returns a reported rate. In a case study on conversational deception, we observe that the reported rate moves substantially with the dialogue setup, the simulated user, the judge, and the evidence bar for non-compliance. These measurement choices substantially change the reported rate and can reorder GenAI system rankings. Because each reported rate is tied to an explicit specification, differences across audits are easier to attribute and interpret. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY). Pagina del lavoro: https://arxiv.org/abs/2608.13840 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:02:02.875Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).