Metraipolis › archive › dc_9ca5a33869ba
Probabilistic Model Checking of Autoregressive Neural Sequence Models — arXiv:2609.00838
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.00838
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T06:33:42.271Z
- SHA-256 of the archived text
0b457bfbdc0ca9eb20489482a6c00a9ec37ed87ffe54db2b697069e717ad3b44- Stable URL of this page
https://metraipolis.com/documento/dc_9ca5a33869ba
How to cite
Probabilistic Model Checking of Autoregressive Neural Sequence Models — arXiv:2609.00838 — arXiv (en), https://arxiv.org/abs/2609.00838 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9ca5a33869ba (Metraipolis archive dc_9ca5a33869ba, sha256 0b457bfbdc0ca9eb20489482a6c00a9ec37ed87ffe54db2b697069e717ad3b44)
BibTeX
@misc{dc9ca5a33869ba,
title = {Probabilistic Model Checking of Autoregressive Neural Sequence Models — arXiv:2609.00838},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.00838},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9ca5a33869ba; sha256 0b457bfbdc0ca9eb20489482a6c00a9ec37ed87ffe54db2b697069e717ad3b44},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00838. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Test-set accuracy is silent on two issues that matter when deploying autoregressive neural sequence models: how much probability mass the system under test (SUT) places on constraint-violating alternatives that are reachable under sampling and what fraction of the input population satisfies a domain requirement. We answer both with probabilistic model checking. The pipeline extracts a discrete-time Markov chain (DTMC) from the SUT's token-by-token generation, verifies formal PCTL specifications with the PRISM model checker, and aggregates the per-input verdicts into a coverage curve over the input space. A soundness theorem establishes the DTMC as an under-approximation, so every verdict yields a certified interval on the SUT's true reachability probability. The coverage built from those verdicts is, therefore, conservative by construction. A counterexample-guided abstraction refinement (CEGAR) loop adaptively tightens the interval, and a maximum-likelihood algorithm extracts the most probable falsifying trace. Two case studies exercise the pipeline. On a GPT-2 computer-aided process-planning (CAPP) model with 100% test accuracy, the pipeline quantifies the probability mass greedy decoding hides, but that is reachable with sampling; and identifies the smallest training fraction at which an ordering requirement holds population-wide, neither of which test accuracy can report. We then verify the SMILES molecular generator with a 50x larger vocabulary. The only change is an external chemical-validity oracle, and the pipeline identifies the gap between structural completeness and chemical validity. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.00838 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T06:15:12.828Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).