Metraipolis › archive › dc_082353141fda
When Agent Metrics Measure Different Things: An Evidence-Grounded Audit of the Praxa AI Pipeline — arXiv:2609.12017
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.12017
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:03:06.367Z
- SHA-256 of the archived text
5f97dd582a4ae692316ffb2124372a42f78865c44f1193fa7a026762218f8078- Stable URL of this page
https://metraipolis.com/documento/dc_082353141fda
How to cite
When Agent Metrics Measure Different Things: An Evidence-Grounded Audit of the Praxa AI Pipeline — arXiv:2609.12017 — arXiv (en), https://arxiv.org/abs/2609.12017 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_082353141fda (Metraipolis archive dc_082353141fda, sha256 5f97dd582a4ae692316ffb2124372a42f78865c44f1193fa7a026762218f8078)
BibTeX
@misc{dc082353141fda,
title = {When Agent Metrics Measure Different Things: An Evidence-Grounded Audit of the Praxa AI Pipeline — arXiv:2609.12017},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.12017},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_082353141fda; sha256 5f97dd582a4ae692316ffb2124372a42f78865c44f1193fa7a026762218f8078},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12017. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agent evaluations can be numerically correct while measuring a different construct from the one implied by their labels. We present a retrospective measurement audit of selected Praxa AI implementation files, historical evaluation artifacts, and operational records. A 139-case offline routing report contains 112 passes and 27 failures despite zero gating failures, because known gaps are explicitly exempted from the gate. An identifier-free export represents 8,843 tool-attempt rows: 8,395 recorded durations and 448 missing values. Of the durations, 121 equal the signed 32-bit maximum and carry abandoned-client labels; inspected database code clamps elapsed lifecycle age. The pooled recorded 99th percentile is 2,147,483,647 ms, versus 38,118.31 ms among server-observed completed calls. This is a stratum contrast, not a treatment effect. In a documented single-trajectory compaction pilot, the reported follow-up input reduction is 94.39%, but the reduction across the trigger and follow-up calls together is 46.54%. We reproduce the descriptive calculations, verify 91 timing statistics through a separate weighted rational-arithmetic implementation, and execute 13 scoring-function and 12 analysis-verifier tests. Finite-completion bounds show how missing durations limit all-row timing statements without imputing values. The contribution is a source-linked case study and reusable verification package for separating gate policy, lifecycle timing, and request-level accounting from broader agent-performance claims. Historical provider runs and the full current pipeline were not independently reproduced; general capability superiority and population-level statistical significance are not established. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.12017 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T02:51:27.849Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).