Metraipolis › archive › dc_678ce3260197
Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems — arXiv:2609.05928
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.05928
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T12:37:03.021Z
- SHA-256 of the archived text
70b5abde456294b745bd1a7230f1bfa410b30029ad443a8a901f6474801d7278- Stable URL of this page
https://metraipolis.com/documento/dc_678ce3260197
How to cite
Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems — arXiv:2609.05928 — arXiv (en), https://arxiv.org/abs/2609.05928 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_678ce3260197 (Metraipolis archive dc_678ce3260197, sha256 70b5abde456294b745bd1a7230f1bfa410b30029ad443a8a901f6474801d7278)
BibTeX
@misc{dc678ce3260197,
title = {Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems — arXiv:2609.05928},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.05928},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_678ce3260197; sha256 70b5abde456294b745bd1a7230f1bfa410b30029ad443a8a901f6474801d7278},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05928. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models now compute correct tax liabilities on over 90% of well-formed cases in statutory benchmarks, which makes them candidates for the tax-advisory and compliance systems that consume such an answer directly. Real legal inputs, however, are frequently defective: required facts are missing, or stated facts contradict one another. Accuracy on clean benchmarks says nothing about how a model behaves then, and a system that computes straight through a defective input returns a confident number with no sign that anything is wrong. This raises two questions: does a model asked to solve a case abstain when the input is defective, and when it does not, can the same model catch the defect when asked instead to verify the input? We study six recent models on SARA-derived tax cases under missing-fact and contradictory-fact perturbations. The strongest models abstain when a fact is missing but compute through injected contradictions, returning the clean-input answer 63-76% of the time with no signal of the conflict; asked instead to verify the same input, they flag most of those contradictions. We wire that verification call into a simple contradiction gate: one extra call that abstains when the model reports a conflict. Across all six models it recovers most of the missed contradiction abstention at a clean-accuracy cost of at most about 5 percentage points, with no training and no external tooling. High accuracy on well-formed inputs is therefore an incomplete measure of reliability, and the detection the solver misses is cheaply recoverable with a single self-check. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.05928 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T12:30:24.999Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).