Metraipolis › archive › dc_2ca5a773fc0e

Why shared attention vectors fail: a case for outcome-indexed tuning — arXiv:2609.08615

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.08615
License (as archived): CC0-1.0
Archived: 2026-10-16T13:58:19.506Z
SHA-256 of the archived text: f70b836423fc0b368dc08c1d9cadc63c43e8c0386f2132d2d519ad810ef72129
Stable URL of this page: https://metraipolis.com/documento/dc_2ca5a773fc0e

How to cite

Why shared attention vectors fail: a case for outcome-indexed tuning — arXiv:2609.08615 — arXiv (en), https://arxiv.org/abs/2609.08615 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2ca5a773fc0e (Metraipolis archive dc_2ca5a773fc0e, sha256 f70b836423fc0b368dc08c1d9cadc63c43e8c0386f2132d2d519ad810ef72129)

BibTeX

@misc{dc2ca5a773fc0e,
  title = {Why shared attention vectors fail: a case for outcome-indexed tuning — arXiv:2609.08615},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08615},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2ca5a773fc0e; sha256 f70b836423fc0b368dc08c1d9cadc63c43e8c0386f2132d2d519ad810ef72129},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08615. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dimensional attention in learning is often implemented as a globally shared attention vector, where each stimulus dimension corresponds to a single scalar. These scalars are learned by models through gradient-descent on error, where predictive features acquire more salience. We show that under multi-outcome learning, where models predict more than one outcome, this shared vector becomes unstable; it collapses to its bounds and prevents the models from learning meaningful attentional tunings for learning and generalization. We address this by introducing an outcome-indexed attentional matrix that converts globally shared attentional tuning into an outcome-indexed representation. We present an analysis of the unstable shared vectors and derive the conditions under which it holds. Empirically, three synthetic experiments benchmark the proposed attention matrices and show that they converge to meaningful representations, something shared attention vectors fail to do. These results suggest that outcome-indexed attentional matrices are a general fix for gradient-based attentional processes, which improves models of learning under multi-outcome conditions.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (q-bio.NC).
Pagina del lavoro: https://arxiv.org/abs/2609.08615 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:39:43.051Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).