Metraipolis › archive › dc_875e4243d283

Tacet: A Language and Type System for Automatic Statistical Validity Accounting — arXiv:2608.27451

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.27451
License (as archived)
CC0-1.0
Archived
2026-10-17T16:16:12.430Z
SHA-256 of the archived text
9b27205abbd71d2bb3a258ec7bdb38f436bb1b38b3982ae337625130c2bfce71
Stable URL of this page
https://metraipolis.com/documento/dc_875e4243d283

How to cite

Tacet: A Language and Type System for Automatic Statistical Validity Accounting — arXiv:2608.27451 — arXiv (en), https://arxiv.org/abs/2608.27451 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_875e4243d283 (Metraipolis archive dc_875e4243d283, sha256 9b27205abbd71d2bb3a258ec7bdb38f436bb1b38b3982ae337625130c2bfce71)
BibTeX
@misc{dc875e4243d283,
  title = {Tacet: A Language and Type System for Automatic Statistical Validity Accounting — arXiv:2608.27451},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.27451},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_875e4243d283; sha256 9b27205abbd71d2bb3a258ec7bdb38f436bb1b38b3982ae337625130c2bfce71},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27451. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Empirical comparisons between systems are a standard form of evidence in computer science research, but few are checked for statistical validity: most are never framed as statistical tests at all. Existing multiple-comparison procedures could control the resulting error, but need inputs (what an analysis examined, and how its observations are arranged) that are not recoverable from a list of p-values. We introduce Tacet, a language in which an analysis declares what it generated, states what it expects to find, and is refused any claim it cannot afford or cannot properly test. Its core calculus T pairs a free estimation sublanguage, carrying a reported footprint and a purity bit that records whether any outcome was consulted in building a value, with a priced claim sublanguage, carrying a wealth transformer, connected only by a mechanism that prices a comparison. A sample selected by reading outcomes sets the purity bit and is recorded as having examined everything it read, permanently, so it can never be granted a one-sided or confirmatory price, without the system ever asking whether the analyst intended to cherry-pick. Whether a comparison is paired or clustered is computed statically from the artifact schema, from declared functional dependencies between key fields alone and before any data is read, and a mechanism that assumes that structure away is refused rather than priced. Because the wealth transformer is antitone in the realized p-value, affordability can be checked before the analysis runs too, turning pre-registration into a typing rule. We prove the metatheory machine-checked in Lean 4 with no admitted gaps, and demonstrate the approach on a reference implementation and two case studies on published artifacts, the SWE-bench Verified leaderboard and BIG-Bench Hard.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-28 in cs.SE).
Categorie arXiv: Programming Languages (cs.PL) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.27451 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T15:36:41.168Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).