Metraipolis › archive › dc_bd3252af06f7
Same Quantity, Different Answer: Numerical Representation Invariance in Language Models — arXiv:2609.25009
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.25009
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T14:34:15.886Z
- SHA-256 of the archived text
dfad85bbd078c7ea37f537a293e29ef96cf7fe2ad3d72885dfce49fa22e367f9- Stable URL of this page
https://metraipolis.com/documento/dc_bd3252af06f7
How to cite
Same Quantity, Different Answer: Numerical Representation Invariance in Language Models — arXiv:2609.25009 — arXiv (en), https://arxiv.org/abs/2609.25009 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd3252af06f7 (Metraipolis archive dc_bd3252af06f7, sha256 dfad85bbd078c7ea37f537a293e29ef96cf7fe2ad3d72885dfce49fa22e367f9)
BibTeX
@misc{dcbd3252af06f7,
title = {Same Quantity, Different Answer: Numerical Representation Invariance in Language Models — arXiv:2609.25009},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.25009},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd3252af06f7; sha256 dfad85bbd078c7ea37f537a293e29ef96cf7fe2ad3d72885dfce49fa22e367f9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25009. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Numerically equivalent word problems should yield the same canonical answer whether a quantity is written as a decimal, fraction, percentage, number word, scientific notation, or an exactly converted unit. We generate 3,600 exact-rational problems and 8,600 prompts spanning five identity-preserving transformation families, and evaluate five open-weight systems. After a fixed syntax audit that normalizes common answer forms without an LLM judge, canonical accuracy is 0.969-0.996, but orbit correctness falls to 0.848-0.981 and orbit invariance to 0.851-0.981; invariant-but-wrong orbits account for at most 0.003. Most of the broad strict-parser collapse arises because multiplication-form scientific notation lies outside the implemented number grammar, illustrating how evaluator interfaces can masquerade as reasoning failures. A distinct semantic pathology remains: Mistral Small 4 scores 0.699 on unit-converted inputs and produces 265 errors differing from the label by exact powers of ten. In a separate 9,000-call experiment that allocates equal calls to the compared arms, representation consensus does not outperform paraphrase consensus on a low-error subset and produces substantially more false alarms. The accompanying ancillary archive contains the frozen benchmark, evaluation and audit records, consensus raw responses, manifests, analysis code, and a one-command paper build. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.25009 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T13:46:15.642Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).