Metraipolis › archive › dc_683c19126ad1

What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity — arXiv:2609.17515

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.17515
License (as archived): CC0-1.0
Archived: 2026-10-15T19:19:29.955Z
SHA-256 of the archived text: 6fbd7d5105e716a723c55bed6431d7f0ea97e691dfdbf316bf8d6516be736186
Stable URL of this page: https://metraipolis.com/documento/dc_683c19126ad1

How to cite

What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity — arXiv:2609.17515 — arXiv (en), https://arxiv.org/abs/2609.17515 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_683c19126ad1 (Metraipolis archive dc_683c19126ad1, sha256 6fbd7d5105e716a723c55bed6431d7f0ea97e691dfdbf316bf8d6516be736186)

BibTeX

@misc{dc683c19126ad1,
  title = {What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity — arXiv:2609.17515},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17515},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_683c19126ad1; sha256 6fbd7d5105e716a723c55bed6431d7f0ea97e691dfdbf316bf8d6516be736186},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17515. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Pruning can reduce the deployment cost of large language models (LLMs), but its impact on context-grounded tool calling remains poorly understood. We systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts (MoE) architectures, together with depth, width, hybrid, and expert pruning methods. After post-pruning supervised fine-tuning (SFT), we evaluate more than 19,500 instances from three smart-home datasets. Beyond aggregate task accuracy, we characterize degradation along two dimensions: action components (i.e., operation, device, argument, and value) and task complexity. Our results show that dense models have narrow safe pruning regions followed by sharp degradation, while MoE models tolerate substantially more pruning. Pruning degrades grounded specificity before schema-level intent, and aggressive dense pruning can induce systematic over-refusal. These findings highlight the importance of evaluating pruning beyond aggregate accuracy when selecting pruned LLMs for reliable tool execution.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.17515 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:32:32.877Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).