Metraipolis › archive › dc_ef781c2f8fc8
Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data — arXiv:2609.35322
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.35322
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:39:02.406Z
- SHA-256 of the archived text
14295309ae735f67fe1133d4d63b27a52d038ac9129209b4769feba526e4b53a- Stable URL of this page
https://metraipolis.com/documento/dc_ef781c2f8fc8
How to cite
Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data — arXiv:2609.35322 — arXiv (en), https://arxiv.org/abs/2609.35322 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef781c2f8fc8 (Metraipolis archive dc_ef781c2f8fc8, sha256 14295309ae735f67fe1133d4d63b27a52d038ac9129209b4769feba526e4b53a)
BibTeX
@misc{dcef781c2f8fc8,
title = {Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data — arXiv:2609.35322},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.35322},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef781c2f8fc8; sha256 14295309ae735f67fe1133d4d63b27a52d038ac9129209b4769feba526e4b53a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35322. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In practice this drift is modeled by a neural network, trained on a loss integrated over time $t$ with a weighting schedule $w(t)$. Along the backward dynamics, and for multi-modal distributions, trajectories commit to modes of the target within a narrow time window, the \textit{speciation time}. In this work, focusing on high-dimensional data, we decompose the integrated loss into its single-time contributions and analyze each at fixed signal-to-noise ratio $\Lambda(t)$: we show that $\Lambda(t)$ sets the rate at which each feature of a multimodal target - the mode directions and their relative weights - is acquired during training. Crucially, at high $\Lambda(t)$ all mode directions are acquired together, on a single timescale insensitive to their amplitudes, while the relative weights are not learned at all. Only near the speciation time, where $\Lambda(t)$ becomes of order one, do all features become learnable, each on its own timescale: the weights are acquired jointly with the directions, and the directions at rates set by their relative amplitudes. For models trained on time-integrated objectives, the learning dynamics is then governed by how much of the weighting effectively sits near the speciation time, which provides insights on $w(t)$ design choices. These results follow from an exact high-dimensional analysis of the training dynamics of unbalanced and hierarchical Gaussian mixtures. Numerical experiments on image and human genome haplotype generation recover the predicted hierarchy of learning timescales in more complex settings.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn).
Pagina del lavoro: https://arxiv.org/abs/2609.35322 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:28.414Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).