Metraipolis › archive › dc_8a3ca562c528

Interpreting Reasoning of Large Language Models via Partial Information Decomposition — arXiv:2610.00571

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.00571
License (as archived): CC0-1.0
Archived: 2026-10-06T12:10:08.526Z
SHA-256 of the archived text: 89f7ffb1446a9f9ad57c236bb91b1cf835a7838b29b442c5e3d6c1f972701829
Stable URL of this page: https://metraipolis.com/documento/dc_8a3ca562c528

How to cite

Interpreting Reasoning of Large Language Models via Partial Information Decomposition — arXiv:2610.00571 — arXiv (en), https://arxiv.org/abs/2610.00571 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8a3ca562c528 (Metraipolis archive dc_8a3ca562c528, sha256 89f7ffb1446a9f9ad57c236bb91b1cf835a7838b29b442c5e3d6c1f972701829)

BibTeX

@misc{dc8a3ca562c528,
  title = {Interpreting Reasoning of Large Language Models via Partial Information Decomposition — arXiv:2610.00571},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.00571},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8a3ca562c528; sha256 89f7ffb1446a9f9ad57c236bb91b1cf835a7838b29b442c5e3d6c1f972701829},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00571. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large reasoning models (LRMs) have achieved substantial improvements in solving complex mathematical problems, but often produce lengthy, repetitive, or erroneous reasoning trajectories. In this work, we introduce a new interpretability framework, SLIDER, to evaluate the quality of the reasoning process. SLIDER leverages an emerging body of work from information theory called Partial Information Decomposition to disentangle the information about the final answer between two consecutive reasoning steps into non-negative components: unique information (in preceding steps or current step), redundant information, and synergistic information. Building on this decomposition, we propose the *Step-wise Repetitive Reasoning Index (Step-RRI)*, a theoretically grounded measure that assesses whether the answer-relevant information in the current step $S_i$ is predominantly redundant with the past steps $S_{<i}$, relative to its unique and synergistic contributions. To evaluate the effectiveness of Step-RRI in detecting repetitiveness, we apply SLIDER to the redundancy class of the PRMBench dataset where Step-RRI improves step-level redundancy identification accuracy by over $10$ points compared to embedding-similarity and information-gain baselines. Next, we define *Trajectory-RRI*, an aggregate measure of repetitiveness for an individual reasoning trajectory. To demonstrate its practical relevance, we show that average Trajectory-RRI strongly correlates with actual reasoning length across QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, and GPT-4.1, motivating its use as a signal for improving reasoning efficiency. Finally, we introduce *Trajectory-RRI-guided data selection for fine-tuning*, demonstrating that selecting training data based on Trajectory-RRI can improve a fine-tuned model's reasoning efficiency while largely preserving its task performance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Information Theory (cs.IT) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.00571 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.085Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).