Metraipolis › archive › dc_4cf7bd9fb218
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models — arXiv:2609.23715
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.23715
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T22:03:46.910Z
- SHA-256 of the archived text
e5fe8ccac64584a4c96779f6e2d548a9a3715a1a93068736269497768589a7cc- Stable URL of this page
https://metraipolis.com/documento/dc_4cf7bd9fb218
How to cite
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models — arXiv:2609.23715 — arXiv (en), https://arxiv.org/abs/2609.23715 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4cf7bd9fb218 (Metraipolis archive dc_4cf7bd9fb218, sha256 e5fe8ccac64584a4c96779f6e2d548a9a3715a1a93068736269497768589a7cc)
BibTeX
@misc{dc4cf7bd9fb218,
title = {Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models — arXiv:2609.23715},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.23715},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4cf7bd9fb218; sha256 e5fe8ccac64584a4c96779f6e2d548a9a3715a1a93068736269497768589a7cc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23715. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal large language models (MLLMs) incur substantial computational overhead due to the reliance on hundreds of visual tokens to represent images. While token pruning has emerged as a promising approach to reduce the inference cost of MLLMs, existing methods typically reassign position embeddings to the retained tokens using either sparse or continuous position embeddings, each introducing distinct limitations. Sparse position embeddings tend to decrease the attention value allocated to visual tokens, thereby degrading the perception capability of MLLMs, whereas continuous position embeddings disrupt the original spatial correspondence of visual tokens, leading to weakened grounding capability. To mitigate this issue, we perform layer-wise analysis of the language decoder and observe that intermediate layers play a critical role for maintaining the grounding capability of MLLMs under token pruning. Based on this observation, we propose a layer-aware position embedding strategy, which switches to sparse position embeddings at grounding-sensitive layers while maintaining continuous position embeddings elsewhere. Extensive experiments across representative pruning methods and diverse benchmarks demonstrate that our approach improves the comprehensive multimodal performance of pruned MLLMs compared with standard sparse and continuous position embeddings. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.23715 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:45:20.944Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).