Metraipolis › archive › dc_b307cd136c5f

Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components — arXiv:2610.41743

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41743
License (as archived)
CC0-1.0
Archived
2026-11-03T07:45:43.202Z
SHA-256 of the archived text
12975127c184f1dba6639e89422bde4864bc1ad321b4b0311a647430d5f56d39
Stable URL of this page
https://metraipolis.com/documento/dc_b307cd136c5f

How to cite

Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components — arXiv:2610.41743 — arXiv (en), https://arxiv.org/abs/2610.41743 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b307cd136c5f (Metraipolis archive dc_b307cd136c5f, sha256 12975127c184f1dba6639e89422bde4864bc1ad321b4b0311a647430d5f56d39)
BibTeX
@misc{dcb307cd136c5f,
  title = {Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components — arXiv:2610.41743},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41743},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b307cd136c5f; sha256 12975127c184f1dba6639e89422bde4864bc1ad321b4b0311a647430d5f56d39},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41743. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Task Vectors (TVs) provide a compact way to steer large language models, but they are typically treated as atomized directions merely extracted from hidden states, leaving their internal functional structure poorly understood. Motivated by prior work that decomposes in-context learning into Task Recognition (TR) and Task Learning (TL), we ask whether a single TV can likewise be decomposed into functionally meaningful components. We formulate a TR/TL-based decomposition from candidate-label unembeddings, separating each TV into TR, TL, and residual components, and try to optimize the TR and TL coefficients using Bayesian optimization for more effective intervention. We evaluate this formulation across multiple model families, classification tasks, and both instruction- and demonstration-based TVs. Across these settings, TR/TL-based optimization identifies intervention directions that outperform the original TVs. Although optimization over random directions can also improve performance, TR/TL-based search yields more reliable gains, particularly for instruction-based TVs. Moreover, isolated interventions on the decomposed components produce distinct logit-level effects that are consistent with their intended functional roles. Together, these results suggest that TVs are not monolithic steering directions, but can be decomposed into functionally meaningful components that support both interpretation and control.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.41743 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:39:52.209Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).