Metraipolis › archive › dc_cc84c4d73aab

MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers — arXiv:2609.33207

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33207
License (as archived)
CC0-1.0
Archived
2026-10-07T22:12:58.737Z
SHA-256 of the archived text
fe9f6017300ae93c5fcecb44d02115dba110e7dd350047ab537cfbf43a73cb41
Stable URL of this page
https://metraipolis.com/documento/dc_cc84c4d73aab

How to cite

MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers — arXiv:2609.33207 — arXiv (en), https://arxiv.org/abs/2609.33207 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cc84c4d73aab (Metraipolis archive dc_cc84c4d73aab, sha256 fe9f6017300ae93c5fcecb44d02115dba110e7dd350047ab537cfbf43a73cb41)
BibTeX
@misc{dccc84c4d73aab,
  title = {MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers — arXiv:2609.33207},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33207},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cc84c4d73aab; sha256 fe9f6017300ae93c5fcecb44d02115dba110e7dd350047ab537cfbf43a73cb41},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33207. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Spiking Vision Transformers (SViTs) are developed as an energy-efficient alternative to conventional ViTs for computer vision tasks at the edge. However, huge parameter counts and complex multi-head self-attention (MHSA) operations make it challenging to achieve high energy efficiency in SViT inference, especially in tightly constrained applications. To maximize efficiency gains of SViT processing, we propose MorphAtt, a novel digital accelerator that expedites SViT inference through streamlined processing. Specifically, it processes MHSA operations using cascaded hardware modules: a Spiking Query-Key-Value generator (SpikeQKV), a low-complexity Spiking Multi-Head Self-Attention engine (SpikeAtten), and Reparameterization Convolution (RepConv) modules. To mitigate traffic congestion in on-chip memory accesses and data reuse, specialized inter-module buffers are integrated within the dataflow. Under synthesis using 32nm CMOS technology, MorphAtt achieves 792-1605 GOPS of throughput, while incurring ~39-55 mW of power consumption and 1.5 mm^2 of area, which lead to 20.3-29.1 TOPS/W of energy efficiency. These results also demonstrate that our MorphAtt offers better performance and efficiency trade-offs than state-of-the-art, thereby enabling highly energy-efficient vision-based AI systems at the edge.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR) ; Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE).
Pagina del lavoro: https://arxiv.org/abs/2609.33207 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:54:30.991Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).