Metraipolis › archive › dc_8341f6519e58
Forward-Free LLM Depth Pruning via Weight Redundancy — arXiv:2609.09883
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.09883
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T07:26:33.128Z
- SHA-256 of the archived text
31623997bd0326aa1a40659f37bc5bc04c82df16d80551a2ba03cde96043e441- Stable URL of this page
https://metraipolis.com/documento/dc_8341f6519e58
How to cite
Forward-Free LLM Depth Pruning via Weight Redundancy — arXiv:2609.09883 — arXiv (en), https://arxiv.org/abs/2609.09883 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8341f6519e58 (Metraipolis archive dc_8341f6519e58, sha256 31623997bd0326aa1a40659f37bc5bc04c82df16d80551a2ba03cde96043e441)
BibTeX
@misc{dc8341f6519e58,
title = {Forward-Free LLM Depth Pruning via Weight Redundancy — arXiv:2609.09883},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.09883},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8341f6519e58; sha256 31623997bd0326aa1a40659f37bc5bc04c82df16d80551a2ba03cde96043e441},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09883. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Depth pruning reduces large language model (LLM) inference cost by removing complete Transformer blocks. Activation-based methods collect hidden states through forward passes on calibration data, while existing forward-free methods score each Transformer block separately without measuring similarity between blocks. We propose Weight-Redundancy Pruning (WRP), a forward-free depth-pruning method that estimates inter-layer redundancy from checkpoint weights to select blocks without calibration data or model forward passes. WRP compares attention output and MLP down-projection weights across layers and combines their pairwise similarities with relative projection-scale information. The resulting all-pairs similarity matrix guides layer grouping and block selection. Across multiple pruning settings, model families, and downstream tasks, WRP consistently outperforms existing forward-free magnitude pruning and approaches the performance of activation-based methods. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2609.09883 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T07:00:16.360Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).