Metraipolis › archive › dc_e997cccc7c87

When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL — arXiv:2608.13385

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.13385
License (as archived)
CC0-1.0
Archived
2026-10-19T04:55:53.327Z
SHA-256 of the archived text
f5b8a5a9552e7ccf11f9cf78652b3f971f2438c31c9870943d073d4da2c0d95b
Stable URL of this page
https://metraipolis.com/documento/dc_e997cccc7c87

How to cite

When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL — arXiv:2608.13385 — arXiv (en), https://arxiv.org/abs/2608.13385 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e997cccc7c87 (Metraipolis archive dc_e997cccc7c87, sha256 f5b8a5a9552e7ccf11f9cf78652b3f971f2438c31c9870943d073d4da2c0d95b)
BibTeX
@misc{dce997cccc7c87,
  title = {When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL — arXiv:2608.13385},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.13385},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e997cccc7c87; sha256 f5b8a5a9552e7ccf11f9cf78652b3f971f2438c31c9870943d073d4da2c0d95b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13385. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Implicit multimodal in-context learning compresses demonstrations into internal interventions, ranging from static task vectors to query-conditioned transformations and attention routing. Despite their common goal, these methods differ substantially in how the intervention depends on the query and where it modifies the model, leaving unclear which additional complexity is necessary for a given task. We propose the Selection--Realization Hypothesis. It views demonstrations as inducing a compact family of internal changes from which the query selects, while the model's computation constrains how the selected change can be implemented. We evaluate this account using controlled multimodal tasks in which query dependence varies without changing the underlying task primitives or prompt format. By contrasting correct demonstrations with matched counterfactuals, we measure the structure of explicit M-ICL and test whether it predicts intervention behavior. We find that the success of a static task vector is closely tied to how much of the demonstration-induced change is shared across queries. Additional intervention complexity becomes useful when explicit M-ICL contains query-specific or distributed structure that a local additive shift cannot recover. These relationships extend to natural VQA benchmarks and support cost-aware method selection without access to test performance. Our results provide a unified empirical theory of when demonstrations can be compressed into a task vector and when a more expressive intervention is warranted.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-14 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.13385 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:46:30.243Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).