Metraipolis › archive › dc_dbe554bbf403

COMPASS: Finding Where Reasoning Lives in Language Models — arXiv:2610.07469

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.07469
License (as archived): CC0-1.0
Archived: 2026-10-07T11:18:40.454Z
SHA-256 of the archived text: 8289c911befd599e0bebb627f780511174317c88a097f72c0aa00eaafc81b53a
Stable URL of this page: https://metraipolis.com/documento/dc_dbe554bbf403

How to cite

COMPASS: Finding Where Reasoning Lives in Language Models — arXiv:2610.07469 — arXiv (en), https://arxiv.org/abs/2610.07469 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dbe554bbf403 (Metraipolis archive dc_dbe554bbf403, sha256 8289c911befd599e0bebb627f780511174317c88a097f72c0aa00eaafc81b53a)

BibTeX

@misc{dcdbe554bbf403,
  title = {COMPASS: Finding Where Reasoning Lives in Language Models — arXiv:2610.07469},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07469},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dbe554bbf403; sha256 8289c911befd599e0bebb627f780511174317c88a097f72c0aa00eaafc81b53a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07469. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Explicitly eliciting reasoning substantially improves LLM performance. Existing approaches require a predefined characterization of reasoning, whether through CoT prompt design, contrastive CoT directions, or via SAE derived reasoning features. For mathematical reasoning with verifiable answers, we show that a much simpler signal suffices, which is the correctness of the model's own direct answer attempts. This signal yields a latent direction that elicits reasoning. This direction is decodable within the activations of most attention heads, but only a small subset of them can be effectively intervened. We introduce COMPASS, an inference-time steering method that identifies these heads using a logit-space attribution score and steers their activations along the correctness direction, requiring only per-head activation statistics. Across three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines we compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70\% fewer generated tokens. Interventions transfer without re-fitting to unseen benchmarks, and ablations show that both the correctness direction and the small set of heads carrying it are necessary, with the effect concentrated in remarkably few heads.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.07469 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:24.509Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).