Metraipolis › archive › dc_b0cc60eb38ef
Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations — arXiv:2609.25745
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.25745
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T15:07:36.466Z
- SHA-256 of the archived text
3b17faeb7cd0966f8f4a92096df0762856cee14c380cda4ce03130f4a145fa26- Stable URL of this page
https://metraipolis.com/documento/dc_b0cc60eb38ef
How to cite
Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations — arXiv:2609.25745 — arXiv (en), https://arxiv.org/abs/2609.25745 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b0cc60eb38ef (Metraipolis archive dc_b0cc60eb38ef, sha256 3b17faeb7cd0966f8f4a92096df0762856cee14c380cda4ce03130f4a145fa26)
BibTeX
@misc{dcb0cc60eb38ef,
title = {Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations — arXiv:2609.25745},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.25745},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b0cc60eb38ef; sha256 3b17faeb7cd0966f8f4a92096df0762856cee14c380cda4ce03130f4a145fa26},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25745. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
RandOpt samples weight-perturbed language models and ensembles top-ranked candidates through plurality voting, but its global perturbation scale ignores heterogeneous module geometry. We propose Modular Norm RandOpt, an architecture-aware sampling method using module-wise natural norms and calibrated scales while preserving selection and voting. It outperforms RandOpt using $3\times$ fewer candidates on Countdown and at least $12\times$ fewer on GSM8K, with corresponding wall-clock savings. Evaluations across seven tasks and three Qwen scales ($0.5$B--$3$B) show higher mean accuracy than RandOpt on Countdown, GSM8K, and MATH-500 at every scale. The gains extend to Llama 3.2 $3$B and Gemma 3 $4$B on Countdown and GSM8K. On Qwen2.5-1.5B, our ensembles also achieve higher mean accuracy than iterative baselines on both tasks at comparable main-run evaluation budgets. On GSM8K, a tail-density diagnostic implies only a $1.2$--$1.8\times$ candidate reduction, while most ensemble improvement is associated with more favorable correct-expert support. These results highlight perturbation geometry as a key design choice for population-efficient, gradient-free search around pretrained models. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.25745 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T14:37:29.413Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).