Metraipolis › archive › dc_ee6e30ff7b6f
Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders — arXiv:2608.14021
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14021
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:20:31.523Z
- SHA-256 of the archived text
50f83be713cbb9ba2ccde45f1a42bbd963c3675e920cc0e04873dff12b558d57- Stable URL of this page
https://metraipolis.com/documento/dc_ee6e30ff7b6f
How to cite
Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders — arXiv:2608.14021 — arXiv (en), https://arxiv.org/abs/2608.14021 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ee6e30ff7b6f (Metraipolis archive dc_ee6e30ff7b6f, sha256 50f83be713cbb9ba2ccde45f1a42bbd963c3675e920cc0e04873dff12b558d57)
BibTeX
@misc{dcee6e30ff7b6f,
title = {Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders — arXiv:2608.14021},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14021},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ee6e30ff7b6f; sha256 50f83be713cbb9ba2ccde45f1a42bbd963c3675e920cc0e04873dff12b558d57},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14021. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Transformer-based sequential recommenders with causal self-attention often rely heavily on the most recent interaction at inference time, but how this behavior is structurally expressed in the representation used for prediction remains unclear. We combine prediction-time diagnostics with norm-based analysis of the full attention block. First, we show that SASRec-style models exhibit highly localized last-item reliance. We then find that, although self-attention aggregates contextual information, residual addition sharply shifts the full-block representation toward same-position contributions, which we term residual dominance. To probe this interpretation, we use inference-time residual scaling as a controlled diagnostic intervention. Changing the residual strength induces a monotonic trade-off between structural mixing and last-item reliance, while reducing residual strength recovers a subset of final-position misses for which representations at non-final positions already rank the ground-truth item correctly. Our results provide a structural account linking extreme last-item reliance to residual dominance at inference time. The code is publicly available. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2608.14021 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:02:02.770Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).