Metraipolis › archive › dc_f282078772bc
When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning — arXiv:2608.14610
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14610
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:02:15.989Z
- SHA-256 of the archived text
56efd0ad523a165733c620a9fabbce6483affb84c31226fce6e409a5c7797771- Stable URL of this page
https://metraipolis.com/documento/dc_f282078772bc
How to cite
When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning — arXiv:2608.14610 — arXiv (en), https://arxiv.org/abs/2608.14610 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f282078772bc (Metraipolis archive dc_f282078772bc, sha256 56efd0ad523a165733c620a9fabbce6483affb84c31226fce6e409a5c7797771)
BibTeX
@misc{dcf282078772bc,
title = {When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning — arXiv:2608.14610},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14610},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f282078772bc; sha256 56efd0ad523a165733c620a9fabbce6483affb84c31226fce6e409a5c7797771},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14610. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Legal reasoning requires identifying which version of a statute applies to a case, typically based on when the legally relevant events occurred rather than when the case is adjudicated. Yet whether large language models (LLMs) can reliably perform such temporal legal reasoning remains largely unexplored. We formalize this task as Temporal Applicable-Law Determination (TALD), and construct a TALD-Bench, a benchmark of 51,956 Chinese civil judgments spanning the enactment of the Civil Code. Evaluating nine advanced LLMs, we identify three key findings. First, models exhibit a strong bias toward newer law, achieving over 70% accuracy on new-law cases but below 25% on old-law cases. Second, models demonstrate substantial knowledge of statutory versions and temporal-effect rules, yet struggle to apply this knowledge to cases. Stronger general reasoning performance does not consistently translate into better TALD accuracy, whereas temporal-law hints yield substantial improvements, suggesting a bottleneck in how models deploy their existing capabilities. Third, temporal-law hints also increase uncertainty during statutory-version reasoning, suggesting that they may help models reconsider their default preference for newer law. Our findings highlight the importance of adapting how LLMs apply existing capabilities, rather than relying solely on improvements in legal knowledge or general reasoning. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.14610 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.711Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).