Metraipolis › archive › dc_e47ac41b28be

FlyWheel: Attention Kernels Built for the TPU — arXiv:2611.01426

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01426
License (as archived)
CC0-1.0
Archived
2026-11-03T11:10:26.856Z
SHA-256 of the archived text
44b8520bab59af06b6401265d86634d4ea3f26d9778b650a7b3a9cd897dbaca6
Stable URL of this page
https://metraipolis.com/documento/dc_e47ac41b28be

How to cite

FlyWheel: Attention Kernels Built for the TPU — arXiv:2611.01426 — arXiv (en), https://arxiv.org/abs/2611.01426 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e47ac41b28be (Metraipolis archive dc_e47ac41b28be, sha256 44b8520bab59af06b6401265d86634d4ea3f26d9778b650a7b3a9cd897dbaca6)
BibTeX
@misc{dce47ac41b28be,
  title = {FlyWheel: Attention Kernels Built for the TPU — arXiv:2611.01426},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01426},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e47ac41b28be; sha256 44b8520bab59af06b6401265d86634d4ea3f26d9778b650a7b3a9cd897dbaca6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01426. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Attention kernels on Tensor Processing Units (TPUs) often underutilize their matrix units (MXUs) because of repeated matrix operand loads, vector-intensive computation, and dependencies that leave MXUs idle. We present FlyWheel, a set of fast TPU kernels for both softmax and linear attention. FlyWheel reformulates softmax and triangular inversion to reduce vector work, while coordinating data layouts to improve matrix operand reuse. It then overlaps independent computation to hide execution latency. We apply these techniques to both softmax attention and Gated DeltaNet on TPU v6e and v7x. FlyWheel speeds up softmax attention by up to $4.14\times$ over the state-of-the-art RPA kernel and $1.80\times$ over Splash Attention, reaching 81% of peak BF16 throughput on v6e. It also speeds up Gated DeltaNet prefill by up to $3.61\times$. Integrated into vLLM serving Qwen3.8-27B, FlyWheel improves prefill throughput by up to $1.56\times$ while maintaining comparable accuracy.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.01426 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:59:52.301Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).