Metraipolis › archive › dc_574a54641b82
Small Models Scout Bottleneck Order for Large-Model Data Control — arXiv:2608.14936
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14936
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:08:13.726Z
- SHA-256 of the archived text
ae8ae22896d66b4a01e27cce15e3ec0a892050cb2bb31ac16c1483700652796e- Stable URL of this page
https://metraipolis.com/documento/dc_574a54641b82
How to cite
Small Models Scout Bottleneck Order for Large-Model Data Control — arXiv:2608.14936 — arXiv (en), https://arxiv.org/abs/2608.14936 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_574a54641b82 (Metraipolis archive dc_574a54641b82, sha256 ae8ae22896d66b4a01e27cce15e3ec0a892050cb2bb31ac16c1483700652796e)
BibTeX
@misc{dc574a54641b82,
title = {Small Models Scout Bottleneck Order for Large-Model Data Control — arXiv:2608.14936},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14936},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_574a54641b82; sha256 ae8ae22896d66b4a01e27cce15e3ec0a892050cb2bb31ac16c1483700652796e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14936. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Small proxy models are commonly used to identify data mixtures for larger-scale training. We ask whether their training trajectories reveal another transferable structure: the order in which larger models should resolve skill bottlenecks. We formulate first-passage skill training, where each monitored skill has a target floor and the objective is to minimize the tokens required to reach all floors. We introduce LogFloor, a closed-loop controller that directs each round toward current bottlenecks, producing phase-ordered resolution trajectories. Across five bAbI skill slices on Qwen2.5-1.5B, LogFloor reduces token cost by 56.2% on average. In 70M-to-12B transfer, three-round replay of a 70M scout path reaches every floor in all eight target runs, saving 30.9% by pair mean, 39.4% in pooled training tokens, and 37.6% under source-cost accounting. On MMLU-control, a frozen scout path succeeds across all eight 12B runs. Collapsing a path to its static marginal mixture or reversing its phase order removes most benefits, while bottleneck labels alone remain partially useful. These results identify phase-ordered bottleneck resolution as a transferable curriculum structure for monitored skill-targeted training. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.14936 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.747Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).