Metraipolis › archive › dc_be7ed8a81a86
MechReasoner: A Simulator and Benchmark for Mechanistic Reasoning in Qualitative Physics — arXiv:2609.34636
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34636
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:58:00.908Z
- SHA-256 of the archived text
67ecfe827b69565b6e524caf8ec6ed456f8a7374982bba3e3fb068d3eb57a0b9- Stable URL of this page
https://metraipolis.com/documento/dc_be7ed8a81a86
How to cite
MechReasoner: A Simulator and Benchmark for Mechanistic Reasoning in Qualitative Physics — arXiv:2609.34636 — arXiv (en), https://arxiv.org/abs/2609.34636 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_be7ed8a81a86 (Metraipolis archive dc_be7ed8a81a86, sha256 67ecfe827b69565b6e524caf8ec6ed456f8a7374982bba3e3fb068d3eb57a0b9)
BibTeX
@misc{dcbe7ed8a81a86,
title = {MechReasoner: A Simulator and Benchmark for Mechanistic Reasoning in Qualitative Physics — arXiv:2609.34636},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34636},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_be7ed8a81a86; sha256 67ecfe827b69565b6e524caf8ec6ed456f8a7374982bba3e3fb068d3eb57a0b9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34636. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
This work introduces MechReasoner, a mechanistic qualitative simulator grounded in confluence-based qualitative physics, together with a benchmark for mechanistic inference. Current large language models (LLMs) generate fluent mechanistic descriptions that do not reliably follow from underlying structural and causal constraints. The benchmark tests whether answers preserve simulator-licensed ambiguity, quantified claims, episode-graph transition evidence, repairs, and trace-support judgments. Its 1,120 items are generated deterministically from admissible interpretation sets, component states, scenario restrictions, confluence constraints, and derivation steps across 18 catalog mechanisms and six task families. Each mechanism undergoes converter checks of structure and topology and behavioral checks against quantitative simulations. GPT-5.5 accuracy decreases as family-specific mechanistic complexity increases, from 76.1% in the lowest-complexity bucket (B1) to 38.0% in the highest-complexity bucket (B4). The negative association remains after controls for rendered-prompt and expected-answer length. These results show that qualitative simulators can support auditable NLP benchmarks for mechanistic inference. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.34636 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:01.616Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).