Metraipolis › archive › dc_14532c939eef

First Learn, Then Memorize: The Spectral Bias of Diffusion Models — arXiv:2609.35377

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35377
License (as archived)
CC0-1.0
Archived
2026-10-07T22:39:06.064Z
SHA-256 of the archived text
47f157538fb2cac8d854798e3d3996626c57a17c0cfab228f2f9afd4e6a8b3fd
Stable URL of this page
https://metraipolis.com/documento/dc_14532c939eef

How to cite

First Learn, Then Memorize: The Spectral Bias of Diffusion Models — arXiv:2609.35377 — arXiv (en), https://arxiv.org/abs/2609.35377 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_14532c939eef (Metraipolis archive dc_14532c939eef, sha256 47f157538fb2cac8d854798e3d3996626c57a17c0cfab228f2f9afd4e6a8b3fd)
BibTeX
@misc{dc14532c939eef,
  title = {First Learn, Then Memorize: The Spectral Bias of Diffusion Models — arXiv:2609.35377},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35377},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_14532c939eef; sha256 47f157538fb2cac8d854798e3d3996626c57a17c0cfab228f2f9afd4e6a8b3fd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35377. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion models trained on a finite dataset first learn to generate novel, high-quality samples and only much later collapse onto their training set. We identify the mechanism behind this separation of timescales and the object that probes it. The training dynamics of the score function are governed---exactly, and at any width---by the Gram matrix of the Neural Tangent Kernel (NTK) evaluated on the noisy training data, so the timescales of generalization and of memorization must be encoded in its spectrum. We show that they are, and that the structure responsible has no analogue in standard kernel settings. The use of multiple noise realizations per sample ($m$ noised copies at a fixed noise level) in the score-matching loss is what restructures the Gram matrix spectrum into two distinct parts. The first, of large eigenvalues, carries the global features of the target distribution and is present already for $m=1$. The second, which the repeated noising creates, consists of the smallest eigenvalues and is supported on eigenvectors aligned with the sample-specific noise directions; it sets a memorization timescale parametrically larger in the training set size $n$. We establish this picture on two fronts. Analytically, we solve the spectrum in the lazy high-dimensional limit for both linear ($n \asymp d$) and polynomial ($n \asymp d^k$) sample complexities, and prove through a bias--variance decomposition that the first bulk minimizes the approximation error while the second drives the error associated with memorization. Empirically, we show the same two-bulk structure in Convolutional NTKs on CelebA and in finite-width U-Nets trained well beyond the lazy regime, and we make the link causal: truncating the Gram matrix at rank $r$ tunes the generalization--memorization transition, and an $L_2$ penalty targeting the second bulk suppresses memorization in feature-learning U-Nets.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn).
Pagina del lavoro: https://arxiv.org/abs/2609.35377 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:28.431Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).