Metraipolis › archive › dc_8763ec458b81

Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM — arXiv:2609.18279

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.18279
License (as archived)
CC0-1.0
Archived
2026-10-15T16:16:24.790Z
SHA-256 of the archived text
530f088c966c91e93204bd64365a5b0848a91a58390adc1135adc0b16befbe6b
Stable URL of this page
https://metraipolis.com/documento/dc_8763ec458b81

How to cite

Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM — arXiv:2609.18279 — arXiv (en), https://arxiv.org/abs/2609.18279 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8763ec458b81 (Metraipolis archive dc_8763ec458b81, sha256 530f088c966c91e93204bd64365a5b0848a91a58390adc1135adc0b16befbe6b)
BibTeX
@misc{dc8763ec458b81,
  title = {Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM — arXiv:2609.18279},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.18279},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8763ec458b81; sha256 530f088c966c91e93204bd64365a5b0848a91a58390adc1135adc0b16befbe6b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18279. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision Transformers (ViTs) have achieved state-of-the-art performance across a range of computer vision tasks, mainly thanks to the self-attention mechanism. However, its complexity, increasing quadratically with the number of tokens, remains the major obstacle to ViT efficiency and deployment at scale. Token merging reduces this cost by aggregating redundant tokens. Yet existing methods are typically evaluated within a single architecture, leaving open whether their effectiveness stems from the merging mechanism itself or from the specific decoder they are paired with. We extend Graph-Guided Token Merging (G2TM), a single module inserted early in a ViT-based network, beyond its original Segmenter setting. We evaluate G2TM across three semantic segmentation frameworks (Segmenter, SETR, EoMT) and three decoder families (Linear, Transformer-, convolution-based), as well as standard ViT image classification. Our results show that G2TM's behavior and accuracy-efficiency trade-off are consistent across every tested architecture for a given backbone size, indicating that its effectiveness is a property of the encoder rather than the decoder. G2TM also generalizes well to image classification, achieving an even smaller degradation in accuracy compared to semantic segmentation. We further find that G2TM's optimal hyperparameters, resulting in a consistent drop in GFLOPs of 22-47% and an increase in throughput by up to 74% for segmentation models on ADE20K dataset, depend primarily on the backbone's pre-training recipe and on the target dataset, rather than on the decoder choice.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.18279 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T15:45:19.767Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).