Metraipolis › archive › dc_a0df993d73a9
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry — arXiv:2609.11163
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11163
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T05:44:21.458Z
- SHA-256 of the archived text
a83e30ff351034b2af81b828aa2049ee5d560a19b1935bbe2afe3558c0c7d0c1- Stable URL of this page
https://metraipolis.com/documento/dc_a0df993d73a9
How to cite
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry — arXiv:2609.11163 — arXiv (en), https://arxiv.org/abs/2609.11163 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a0df993d73a9 (Metraipolis archive dc_a0df993d73a9, sha256 a83e30ff351034b2af81b828aa2049ee5d560a19b1935bbe2afe3558c0c7d0c1)
BibTeX
@misc{dca0df993d73a9,
title = {LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry — arXiv:2609.11163},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11163},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a0df993d73a9; sha256 a83e30ff351034b2af81b828aa2049ee5d560a19b1935bbe2afe3558c0c7d0c1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11163. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Structured pruning of large language models (LLMs) offers hardware-efficient compression, yet existing methods require calibration data, gradient computation, or large auxiliary policy networks at pruning time. LILA (\emph{Latent-Informed Layer Analysis}) scores neuron importance via the Kolmogorov--Smirnov (KS) distance between empirical singular value distributions of the full and neuron-ablated feed-forward network (FFN) weight matrix, providing a closed-form spectral rule requiring no training, calibration data, or auxiliary network. Without any fine-tuning, LILA surpasses PruneNet (45M-parameter RL policy) by 1.57~pp in zero-shot accuracy on LLaMA-2-7B at 25\% sparsity, and outperforms WikiText-2-calibrated SliceGPT by up to 6.0~pp across all sparsity levels, while preserving the original architecture. After one epoch of LoRA recovery fine-tuning, LILA achieves highly competitive performance, matching the heavily calibrated SliceGPT baseline to within a 0.48~pp margin across LLaMA-2-7B and Phi-2, despite using zero calibration data. A Neural Tangent Kernel analysis confirms a 22$\times$ reduction in functional distortion versus random pruning, providing theoretical grounding for the spectral importance criterion. Finally, extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.11163 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:23:57.040Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).