Metraipolis › archive › dc_ffc5c9ea97cc

Improving the Chain-of-Thought Compressibility of Pruned Reasoning Models via Long--Short Contrastive Pruning — arXiv:2610.34336

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.34336
License (as archived)
CC0-1.0
Archived
2026-10-27T08:01:20.291Z
SHA-256 of the archived text
73272b2a36076258ea397b23c4c651353c9452883da3a5fce996c913cd32efb8
Stable URL of this page
https://metraipolis.com/documento/dc_ffc5c9ea97cc

How to cite

Improving the Chain-of-Thought Compressibility of Pruned Reasoning Models via Long--Short Contrastive Pruning — arXiv:2610.34336 — arXiv (en), https://arxiv.org/abs/2610.34336 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffc5c9ea97cc (Metraipolis archive dc_ffc5c9ea97cc, sha256 73272b2a36076258ea397b23c4c651353c9452883da3a5fce996c913cd32efb8)
BibTeX
@misc{dcffc5c9ea97cc,
  title = {Improving the Chain-of-Thought Compressibility of Pruned Reasoning Models via Long--Short Contrastive Pruning — arXiv:2610.34336},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.34336},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffc5c9ea97cc; sha256 73272b2a36076258ea397b23c4c651353c9452883da3a5fce996c913cd32efb8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34336. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large reasoning models are costly to serve because they are large and because they generate long chains of thought (CoT). Weight pruning and CoT compression address these two costs separately, but they interfere in that pruning adds redundant reasoning on problems the model still solves and incomplete reasoning on those it no longer solves, and on harder problems makes the model less compressible by subsequent length-preference optimization. We trace this to a blind spot on each side: pruning criteria do not distinguish necessary from redundant reasoning, and preference-based CoT compression receives no training signal on problems a pruned model always or never solves. We propose Contrastive Set-Difference (CSD), which identifies neurons enriched in redundant reasoning by contrasting their activations on long and short correct solutions of the same problem and prunes them preferentially, and we extend preference pairs to the problems a pruned model always or never solves. Pruning 20\% of DeepSeek-R1-Distill-Qwen-7B, the combination uses 8--22\% fewer thinking tokens on four math benchmarks than pruning with reasoning-aware calibration (RAC) followed by the same compression, with accuracy within 0.5 points.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.34336 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.894Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).