Metraipolis › archive › dc_52fbfcac85db

Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components — arXiv:2610.41743

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41743
License (as archived): CC0-1.0
Archived: 2026-11-02T07:23:26.520Z
SHA-256 of the archived text: 635a00bbdd0ac761a967e34f314e1f5d2a9d06c672601b62a81949d3e4d51476
Stable URL of this page: https://metraipolis.com/documento/dc_52fbfcac85db

How to cite

Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components — arXiv:2610.41743 — arXiv (en), https://arxiv.org/abs/2610.41743 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_52fbfcac85db (Metraipolis archive dc_52fbfcac85db, sha256 635a00bbdd0ac761a967e34f314e1f5d2a9d06c672601b62a81949d3e4d51476)

BibTeX

@misc{dc52fbfcac85db,
  title = {Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components — arXiv:2610.41743},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41743},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_52fbfcac85db; sha256 635a00bbdd0ac761a967e34f314e1f5d2a9d06c672601b62a81949d3e4d51476},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41743. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Task Vectors (TVs) provide a compact way to steer large language models, but they are typically treated as atomized directions merely extracted from hidden states, leaving their internal functional structure poorly understood. Motivated by prior work that decomposes in-context learning into Task Recognition (TR) and Task Learning (TL), we ask whether a single TV can likewise be decomposed into functionally meaningful components. We formulate a TR/TL-based decomposition from candidate-label unembeddings, separating each TV into TR, TL, and residual components, and try to optimize the TR and TL coefficients using Bayesian optimization for more effective intervention. We evaluate this formulation across multiple model families, classification tasks, and both instruction- and demonstration-based TVs. Across these settings, TR/TL-based optimization identifies intervention directions that outperform the original TVs. Although optimization over random directions can also improve performance, TR/TL-based search yields more reliable gains, particularly for instruction-based TVs. Moreover, isolated interventions on the decomposed components produce distinct logit-level effects that are consistent with their intended functional roles. Together, these results suggest that TVs are not monolithic steering directions, but can be decomposed into functionally meaningful components that support both interpretation and control.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.41743 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:17:12.158Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).