Metraipolis › archive › dc_5be05a01a925

FLARE++: Low-rank attention with dynamic attention routing — arXiv:2608.11519

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11519
License (as archived)
CC0-1.0
Archived
2026-10-19T05:39:48.617Z
SHA-256 of the archived text
3a3b287bdcdd3e2c84aeaaa66933fc628d9ffbdf59a3a009b80e70f5af9fa823
Stable URL of this page
https://metraipolis.com/documento/dc_5be05a01a925

How to cite

FLARE++: Low-rank attention with dynamic attention routing — arXiv:2608.11519 — arXiv (en), https://arxiv.org/abs/2608.11519 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5be05a01a925 (Metraipolis archive dc_5be05a01a925, sha256 3a3b287bdcdd3e2c84aeaaa66933fc628d9ffbdf59a3a009b80e70f5af9fa823)
BibTeX
@misc{dc5be05a01a925,
  title = {FLARE++: Low-rank attention with dynamic attention routing — arXiv:2608.11519},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11519},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5be05a01a925; sha256 3a3b287bdcdd3e2c84aeaaa66933fc628d9ffbdf59a3a009b80e70f5af9fa823},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11519. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Full self-attention is a strong token mixer for PDE surrogates on irregular domains, but its quadratic cost limits its use on large problems. Latent-space attention methods such as PerceiverIO, Transolver, and FLARE (Fast Low-rank Attention Routing Engine) avoid that cost by routing attention among $N$ tokens through $M\ll N$ learned latents. They compress by dot-product matching of the input tokens against $M$ learned query tokens or projection weights: once trained, the same learned templates serve every input. We remove this restriction with FLARE++, a low-rank attention architecture with input-conditioned routing queries. FLARE++ uses FLARE's own encoder to map the $N$ input tokens to $M$ query tokens, which correct the learned queries. The adapted queries then determine how that same input is compressed and redistributed. This preserves FLARE's explicit low-rank factorization and linear $\mathcal O(NM)$ complexity, and expresses the complete routing operation with standard scaled dot-product attention (SDPA) calls alone. We also provide a multi-GPU context-parallel implementation that shards input tokens across devices without ever gathering the full token sequence on one of them. FLARE++ reduces FLARE's error by $25\%$ on average across five standard PDE benchmarks, achieving the lowest errors among the efficient models compared. The gains persist on industrial-scale DrivAerML aerodynamics and on Long Range Arena, where average accuracy rises by $3.5\%$ over fixed-query FLARE. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.11519 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:18:32.470Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).