Metraipolis › archive › dc_55ec0d857cb4
Amortizing Scaling Law Construction Costs — arXiv:2609.05016
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.05016
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:28:59.920Z
- SHA-256 of the archived text
d49f61ddceffc43e2f7dd6266f1576489b66a25a248bee1a99cf0b2f3b9bbafc- Stable URL of this page
https://metraipolis.com/documento/dc_55ec0d857cb4
How to cite
Amortizing Scaling Law Construction Costs — arXiv:2609.05016 — arXiv (en), https://arxiv.org/abs/2609.05016 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_55ec0d857cb4 (Metraipolis archive dc_55ec0d857cb4, sha256 d49f61ddceffc43e2f7dd6266f1576489b66a25a248bee1a99cf0b2f3b9bbafc)
BibTeX
@misc{dc55ec0d857cb4,
title = {Amortizing Scaling Law Construction Costs — arXiv:2609.05016},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.05016},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_55ec0d857cb4; sha256 d49f61ddceffc43e2f7dd6266f1576489b66a25a248bee1a99cf0b2f3b9bbafc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05016. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Scaling laws guide the design choices for training large foundation models, but deriving them involves training an exhaustive grid over hyperparameters, token budgets, and parameter counts, which is computationally expensive. Fitting a scaling law, however, only requires the best-loss frontier across compute scales, discarding most of the trained configurations. We propose a framework for efficient scaling law construction that formulates data collection as a Bayesian optimization problem, and introduce metrics for comparing scaling law fitting methods under constrained compute budgets. We find that progressively expanding the compute budget during acquisition, mirroring the compute-ordered evaluation of configurations in practice, substantially improves recovery efficiency. Augmenting the observed configurations with surrogate-fantasized evaluations then recovers the broader experimental grid, allowing accurate scaling law fitting without training every configuration. Together, these can closely match scaling law fits over a full dense grid at computational savings of up to $10\text{--}100\times$.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.05016 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:18:11.562Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).