Metraipolis › archive › dc_cb0e08cbd241

One Verdict Is Not Enough: Regression Testing Learned World Models and Their Test Harnesses — arXiv:2610.41130

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41130
License (as archived)
CC0-1.0
Archived
2026-11-02T04:44:36.408Z
SHA-256 of the archived text
24d846649e363b19e124301eeac84e623b688f4f34287e5e7e6a28d29c246f20
Stable URL of this page
https://metraipolis.com/documento/dc_cb0e08cbd241

How to cite

One Verdict Is Not Enough: Regression Testing Learned World Models and Their Test Harnesses — arXiv:2610.41130 — arXiv (en), https://arxiv.org/abs/2610.41130 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cb0e08cbd241 (Metraipolis archive dc_cb0e08cbd241, sha256 24d846649e363b19e124301eeac84e623b688f4f34287e5e7e6a28d29c246f20)
BibTeX
@misc{dccb0e08cbd241,
  title = {One Verdict Is Not Enough: Regression Testing Learned World Models and Their Test Harnesses — arXiv:2610.41130},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41130},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cb0e08cbd241; sha256 24d846649e363b19e124301eeac84e623b688f4f34287e5e7e6a28d29c246f20},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41130. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Learned world models ship as software components whose model versions, preprocessing, and test harnesses change between releases. Work on model updates already counts per-sample regressions, or negative flips, that an accuracy check hides. Two problems remain: a test can pass under one configuration and fail under another, and a harness fault can produce erroneous scores and wrong verdicts without a crash. We present TetraState, a regression-testing protocol for paired possible/impossible video tests. Each test runs under four fixed model-native configurations; its number of correct verdicts assigns one of four outcome classes. Inconclusive tests and failed executions are reported separately alongside the verdict rate. Revisions are compared by per-test outcome transitions, and the harness is versioned as part of the system under test. On all 506 IntPhys 2 Main tests and two heterogeneous systems, Cosmos 3 Nano and V-JEPA 2, two V-JEPA 2 harness revisions move modal accuracy by only 0.2 and 2.6 points yet change 11 and 214 test labels; in the second, accuracy rises although fewer tests are judged correctly, and 43 regressions keep the modal verdict, so a modal-verdict flip check misses them. These results outline a research agenda on configuration-level regression oracles for learned components.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.41130 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T04:42:12.448Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).