Metraipolis › archive › dc_3dfd66cf88a4

Low-Rank Hierarchical Merging for Efficient Long-to-Short Reasoning — arXiv:2611.03834

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.03834
License (as archived): CC0-1.0
Archived: 2026-11-04T18:22:05.284Z
SHA-256 of the archived text: feb214f58677af7ecee1716eb52425a3fc06a14eea76b2d90a03177de4535944
Stable URL of this page: https://metraipolis.com/documento/dc_3dfd66cf88a4

How to cite

Low-Rank Hierarchical Merging for Efficient Long-to-Short Reasoning — arXiv:2611.03834 — arXiv (en), https://arxiv.org/abs/2611.03834 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3dfd66cf88a4 (Metraipolis archive dc_3dfd66cf88a4, sha256 feb214f58677af7ecee1716eb52425a3fc06a14eea76b2d90a03177de4535944)

BibTeX

@misc{dc3dfd66cf88a4,
  title = {Low-Rank Hierarchical Merging for Efficient Long-to-Short Reasoning — arXiv:2611.03834},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03834},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3dfd66cf88a4; sha256 feb214f58677af7ecee1716eb52425a3fc06a14eea76b2d90a03177de4535944},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03834. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-to-Short (L2S) model merging seeks to combine the strong reasoning capability of long chain-of-thought models with the concise style of base models, but existing methods that rely on linear interpolation and first-order activation statistics fail to capture how the loss landscape responds to parameter displacement, leading to degraded accuracy on hard tasks. We propose Low-Rank Hierarchical Merging (LHM), a training-free framework that replaces heuristic statistics with lightweight second-order curvature information. LHM consists of three components: (1) a low-rank Hessian approximation that efficiently estimates per-layer Hessian traces via Hutchinson's estimator on random inputs, eliminating the need for task-specific data; (2) a Layer Second-Order Sensitivity (LSOS) index that combines Hessian trace with task-vector norm to quantify per-layer fusion difficulty; and (3) a Dual-Constraint Coefficient Mapping (DCCM) that converts LSOS scores into layer-adaptive merge coefficients while jointly satisfying a fusion-error bound and a global length-reduction target. Experiments across six mathematical benchmarks on 1.5B and 7B models show that LHM outperforms state-of-the-art methods such as ACM-TIES and ORCA, achieving over 4\% accuracy gain and up to 93.1\% reduction in output length relative to the long-CoT model.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.03834 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:21:13.495Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).