Metraipolis › archive › dc_cb41e5478bb4

Interpreting Model Reasoning through Behavioral Counterfactuals — arXiv:2610.30630

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.30630
License (as archived)
CC0-1.0
Archived
2026-10-25T08:34:33.974Z
SHA-256 of the archived text
be666c4883e1fa2c7c4c84592d84e0568cc7fab24bf27bfc0b921222a2a838e2
Stable URL of this page
https://metraipolis.com/documento/dc_cb41e5478bb4

How to cite

Interpreting Model Reasoning through Behavioral Counterfactuals — arXiv:2610.30630 — arXiv (en), https://arxiv.org/abs/2610.30630 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cb41e5478bb4 (Metraipolis archive dc_cb41e5478bb4, sha256 be666c4883e1fa2c7c4c84592d84e0568cc7fab24bf27bfc0b921222a2a838e2)
BibTeX
@misc{dccb41e5478bb4,
  title = {Interpreting Model Reasoning through Behavioral Counterfactuals — arXiv:2610.30630},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.30630},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cb41e5478bb4; sha256 be666c4883e1fa2c7c4c84592d84e0568cc7fab24bf27bfc0b921222a2a838e2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30630. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Faithfully understanding the internal process that occurs as models form their chain of thought is an important problem with the rise in model capabilities. However, this remains challenging due to the variance in sampled responses, making it difficult to isolate the effects of interventions and compare model reasoning systematically. We propose a method for localizing and analyzing how a given intervention affects the model's reasoning through behavioral counterfactuals: given a fixed response, how does introducing an intervention to the prompt change the next-token distributions? We train a module to edit the residual stream such that it minimizes the KL divergence between the output distributions for a fixed response under a source prompt and its counterfactual target prompt. We find that matching the output distributions recovers the counterfactual behavior more effectively than standard activation reconstruction objectives and validate that attribution of the trained module outputs is effective for identifying causally important locations for counterfactual recovery. The attribution maps and visualizations of the downstream effects of model edits provide an interface for generating hypotheses about the original model's computation, which can then be tested with activation patching. In a case study on Gemma-4-E4B-it and DeepSeek-R1-0528-Qwen3-8B, we investigate how a suggestion influences model reasoning even when it is not verbally acknowledged. Our analysis identifies representations at the question statement and filler tokens that mediate a substantial part of the suggestion's influence, with downstream effects such as changing the framing of the justification or even the final answer.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.30630 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T08:31:13.173Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).