Metraipolis › archive › dc_79bafbbc69b9

Memory Bandwidth, Not FLOPs: Profiling and Accelerating Query-Based Segmentation Decoders on Edge GPUs — arXiv:2610.22377

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22377
License (as archived)
CC0-1.0
Archived
2026-10-19T08:44:19.695Z
SHA-256 of the archived text
3016d38065461f9619719e078a52fc2c99687348818659bfee91dcc3d93ae9cc
Stable URL of this page
https://metraipolis.com/documento/dc_79bafbbc69b9

How to cite

Memory Bandwidth, Not FLOPs: Profiling and Accelerating Query-Based Segmentation Decoders on Edge GPUs — arXiv:2610.22377 — arXiv (en), https://arxiv.org/abs/2610.22377 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_79bafbbc69b9 (Metraipolis archive dc_79bafbbc69b9, sha256 3016d38065461f9619719e078a52fc2c99687348818659bfee91dcc3d93ae9cc)
BibTeX
@misc{dc79bafbbc69b9,
  title = {Memory Bandwidth, Not FLOPs: Profiling and Accelerating Query-Based Segmentation Decoders on Edge GPUs — arXiv:2610.22377},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22377},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_79bafbbc69b9; sha256 3016d38065461f9619719e078a52fc2c99687348818659bfee91dcc3d93ae9cc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22377. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Query-based segmentation models achieve strong performance even under real-time inference on workstation GPUs. On edge devices, however, we show their transformer decoders are deeply memory-bandwidth-bound, a bottleneck invisible to standard FLOP analysis that worsens with input resolution and limited DRAM bandwidth. An FP16 roofline analysis with tensor-core GEMMs active and hardware DRAM counters shows that the non-tensor-core aggregates of FastInst and Mask2Former reach only $2.5\%$ to $6\%$ of the CUDA-core ridge yet account for $53\%$ to $70\%$ of decoder time and $61\%$ to $71\%$ of decoder bytes. The largest individual kernels sustain up to $90\%$ of nominal peak DRAM bandwidth, effectively saturating the memory bus. The bottleneck is therefore not primarily low GEMM intensity, but cross-kernel activation traffic. CompactInst addresses this bottleneck using parameter-free spatial pooling at $4{\times}$, which reduces the bytes moved within the transformer decoder, while a two-convolution refinement head recovers spatial detail. Under TensorRT FP16 on Jetson Orin Nano, CompactInst delivers decoder speedups from $2.35{\times}$ to $4.74{\times}$ and model speedups from $1.57{\times}$ to $2.62{\times}$ as input resolution grows, while closely matching accuracy ($-0.4$ AP on COCO val2017, $+0.2$ AP on Cityscapes test). A heavier variant with $2.57{\times}$ more decoder FLOPs still runs its decoder $1.74{\times}$ faster on the most bandwidth-constrained device, confirming at the deployment level that decoder bytes, not FLOPs, govern latency.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR) ; Machine Learning (cs.LG); Performance (cs.PF); Image and Video Processing (eess.IV).
Pagina del lavoro: https://arxiv.org/abs/2610.22377 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:54.815Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).