Metraipolis › archive › dc_339b0df5bb07

Rethinking Vision Architectures with Gated Linear Attention and KAN — arXiv:2609.22506

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.22506
License (as archived)
CC0-1.0
Archived
2026-10-09T21:59:11.974Z
SHA-256 of the archived text
3e0334ecc02aaa1b0e2bd89d8b31105834e07c70c7736e2a2ebe4140bbdc87b5
Stable URL of this page
https://metraipolis.com/documento/dc_339b0df5bb07

How to cite

Rethinking Vision Architectures with Gated Linear Attention and KAN — arXiv:2609.22506 — arXiv (en), https://arxiv.org/abs/2609.22506 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_339b0df5bb07 (Metraipolis archive dc_339b0df5bb07, sha256 3e0334ecc02aaa1b0e2bd89d8b31105834e07c70c7736e2a2ebe4140bbdc87b5)
BibTeX
@misc{dc339b0df5bb07,
  title = {Rethinking Vision Architectures with Gated Linear Attention and KAN — arXiv:2609.22506},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22506},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_339b0df5bb07; sha256 3e0334ecc02aaa1b0e2bd89d8b31105834e07c70c7736e2a2ebe4140bbdc87b5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22506. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision Transformers devote most of their parameters to MLPs for channel mixing, but still rely on quadratic multi-head self-attention for token interactions. While linear attention fixes the complexity problem, bringing it down to O(N), it is usually just paired with the same fixed-activation MLP as before. Kolmogorov-Arnold Networks take a different approach, placing learnable univariate functions on the edges instead. However, existing vision KANs either retain standard attention or remove attention entirely, so the two ideas have not been effectively combined. We introduce LKAT (Linear Kolmogorov-Arnold Transformer) to close this gap: an isotropic ViT-style encoder that couples chunk-wise Gated Linear Attention with a two-layer KAN feed-forward block, backed by an I/O-aware fused RBF-KAN kernel to make radial-basis grid functions efficient in practice. Under a shared DeiT-style training recipe, LKAT-B outperforms ViT-B/16, ViT-5-B, and Mixer-B/16 on ImageNet-100, while Tiny, Small, and Base variants scale consistently on CIFAR-10/100. ImageNet-100 pretraining also transfers effectively to CIFAR fine-tuning, suggesting that gated linear attention and KAN-based radial basis functions provide complementary inductive biases for mid-scale visual representation learning. Code: this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.22506 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T21:45:20.794Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).