Metraipolis › archive › dc_5295e9b4e84b
Diagnosing Sampled LLM Reasoning in Formal Geometry: Coverage, Realization, and Validity Evidence — arXiv:2609.32924
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32924
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:56:18.121Z
- SHA-256 of the archived text
0ed4efd3a3ca0d3eec1b327605f8cb29ab991b4b2e55209758db9302e3b7379b- Stable URL of this page
https://metraipolis.com/documento/dc_5295e9b4e84b
How to cite
Diagnosing Sampled LLM Reasoning in Formal Geometry: Coverage, Realization, and Validity Evidence — arXiv:2609.32924 — arXiv (en), https://arxiv.org/abs/2609.32924 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5295e9b4e84b (Metraipolis archive dc_5295e9b4e84b, sha256 0ed4efd3a3ca0d3eec1b327605f8cb29ab991b4b2e55209758db9302e3b7379b)
BibTeX
@misc{dc5295e9b4e84b,
title = {Diagnosing Sampled LLM Reasoning in Formal Geometry: Coverage, Realization, and Validity Evidence — arXiv:2609.32924},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32924},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5295e9b4e84b; sha256 0ed4efd3a3ca0d3eec1b327605f8cb29ab991b4b2e55209758db9302e3b7379b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32924. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Repeated sampling can reveal a correct numerical answer without yielding either a reliable system output or a supported derivation. We present Coverage, Realization, and Validity Evidence (CRV), an evaluation protocol for sampled large language model (LLM) reasoning over formal geometry states. Coverage is answer availability, realization is readout accuracy on the frozen candidate pool, and validity evidence is a label-blinded critic judgment of derivational support rather than a proof certificate. CRV freezes each candidate pool before comparing readouts and analyzes covered failures by correct-answer multiplicity and within-problem discrimination. On HardShift441, a 441-problem set for which a reference solver leaves 406 problems unsolved, a LoRA-adapted Qwen2.5-7B generator obtains 24.2% average single-sample accuracy and 68.9% pass@16, whereas verifier-weighted self-consistency (WSC) reaches 38.0%. Readout accuracy is particularly low when the correct answer occurs only once or twice in the pool. In a separate constructed audit of 195 covered problems, the critic labels 12 correct-answer representatives as supported, 181 as refuted, and two as uncertain. These results show that coverage, realization, and validity evidence from the critic are distinct quantities and should be reported separately. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.32924 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:01.114Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).