Metraipolis › archive › dc_bdeef4f6d916
PFA: Attention Can Be Progressively Focused in Long-Sequence Generative Recommendation — arXiv:2610.39273
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.39273
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-01T02:39:57.423Z
- SHA-256 of the archived text
bae46219dbbcdbfa4ff1eb0cf881bedd72e925aca70dd1bb5370776286fad945- Stable URL of this page
https://metraipolis.com/documento/dc_bdeef4f6d916
How to cite
PFA: Attention Can Be Progressively Focused in Long-Sequence Generative Recommendation — arXiv:2610.39273 — arXiv (en), https://arxiv.org/abs/2610.39273 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bdeef4f6d916 (Metraipolis archive dc_bdeef4f6d916, sha256 bae46219dbbcdbfa4ff1eb0cf881bedd72e925aca70dd1bb5370776286fad945)
BibTeX
@misc{dcbdeef4f6d916,
title = {PFA: Attention Can Be Progressively Focused in Long-Sequence Generative Recommendation — arXiv:2610.39273},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.39273},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bdeef4f6d916; sha256 bae46219dbbcdbfa4ff1eb0cf881bedd72e925aca70dd1bb5370776286fad945},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39273. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Long user behavioral sequences improve the performance of generative recommendation (GR), but computing full attention over long histories during autoregressive beam search decoding incurs substantial online inference latency. To accelerate long-sequence decoding, existing approaches typically rely on static history compression or dynamic top-k sparse attention; however, the former risks losing critical semantics, whereas the latter incurs significant overhead from repeated token selection and non-contiguous memory access. In this work, we thoroughly analyze attention patterns during GR decoding and reveal two key properties: (i) progressive attention focusing, where attention progressively focuses on a small subset of critical historical tokens; and (ii) cross-step ranking consistency, where their importance rankings remain relatively stable across subsequent decoding steps. Motivated by these observations, we propose Progressively Focused Attention (PFA), an efficient sparse decoding framework for long-sequence GR. PFA retains full attention at the initial decoding step to explore global historical preferences. It then establishes a shared importance ranking through cross-beam weighted aggregation and reorders the key-value cache of the selected tokens into a contiguous memory layout. Subsequent decoding steps reuse this ranking and geometrically reduce the token budget, preserving critical semantics while lowering computational and memory access overhead. Extensive experiments on two public benchmarks and a large-scale industrial dataset demonstrate that PFA outperforms state-of-the-art baselines. A 7-day online A/B test on a large-scale e-commerce advertising platform further confirms that PFA delivers substantial business gains while maintaining low inference latency. Our code is publicly available at this https URL . Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.IR, nuovi lavori del giorno). Categorie arXiv: Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2610.39273 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-01T02:16:28.222Z dalla pagina https://arxiv.org/list/cs.IR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).