Metraipolis › archive › dc_10a96ace692e
The Calibration Floor: Format Repair Can Masquerade as Self-Correction at Small-to-Mid Scale — arXiv:2608.04355
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04355
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:24:37.203Z
- SHA-256 of the archived text
351e19fa8f38eb34590254f53f96fd7bb2806aa2967a3d3366fa561a2da6d0c6- Stable URL of this page
https://metraipolis.com/documento/dc_10a96ace692e
How to cite
The Calibration Floor: Format Repair Can Masquerade as Self-Correction at Small-to-Mid Scale — arXiv:2608.04355 — arXiv (en), https://arxiv.org/abs/2608.04355 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_10a96ace692e (Metraipolis archive dc_10a96ace692e, sha256 351e19fa8f38eb34590254f53f96fd7bb2806aa2967a3d3366fa561a2da6d0c6)
BibTeX
@misc{dc10a96ace692e,
title = {The Calibration Floor: Format Repair Can Masquerade as Self-Correction at Small-to-Mid Scale — arXiv:2608.04355},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04355},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_10a96ace692e; sha256 351e19fa8f38eb34590254f53f96fd7bb2806aa2967a3d3366fa561a2da6d0c6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04355. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Accuracy changes after language-model self-revision are usually interpreted as changes in reasoning. We show this can fail at the answer-extraction boundary, and test the failure causally rather than only observationally. Across Qwen3.5 (0.8B-9B), Gemma-4-12B, and two frontier models via API (Tencent Hy3, Nvidia Nemotron-3-Ultra-550B) in 29 primary cells plus a frontier arm, we decompose the always-revise accuracy shift into a content margin (both answers parseable) and format-recovery/loss margins (parseability changes). On 12 cells with meaningful unparseable-answer rates, format effects exceed content effects (Wilcoxon p=1.7e-3). To test this causally, we force already-generated reasoning through grammar-constrained decoding so every answer is parseable by construction: across 14 cells this closes a median 71% of the gap between the naive total effect and the content-margin estimate, with two cells converging exactly and a residual on the two largest-effect cells reported rather than dismissed. A clustered model confirms floor-scale (0.8B/2B) models have far higher odds of content-level change and harm than capable-scale models (p<1e-7). Replicating a cited confidence-gating protocol verbatim on Qwen3.5 does not reproduce its reported gain and shows the same near-zero content margin. A frontier check on much larger models shows format-dominance intensifying with scale: content margin is exactly zero in all 5 cells despite total effects up to +0.275, though this arm is lower-powered. The calibration-floor criterion on the content margin reveals a squeeze: floor-scale cells have headroom but insufficient signal, capable-scale cells have signal but little headroom; only one cell is marginally viable, with negligible sealed-holdout gain. Content is a minority share of what the field has measured as self-correction. We release the instrument, code, and derived results. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.04355 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:13:23.164Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).