Metraipolis › archive › dc_676776fa6d79
Model Collapse Is Avoidable Even as Real Data Vanishes: A Sharp Criterion for Iterative Gaussian Estimation — arXiv:2611.00259
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00259
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:39:22.752Z
- SHA-256 of the archived text
db0f1688e0367a94d25d1f8472ce09d746ccac4e4c0676a2fbef31c9f339e2e3- Stable URL of this page
https://metraipolis.com/documento/dc_676776fa6d79
How to cite
Model Collapse Is Avoidable Even as Real Data Vanishes: A Sharp Criterion for Iterative Gaussian Estimation — arXiv:2611.00259 — arXiv (en), https://arxiv.org/abs/2611.00259 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_676776fa6d79 (Metraipolis archive dc_676776fa6d79, sha256 db0f1688e0367a94d25d1f8472ce09d746ccac4e4c0676a2fbef31c9f339e2e3)
BibTeX
@misc{dc676776fa6d79,
title = {Model Collapse Is Avoidable Even as Real Data Vanishes: A Sharp Criterion for Iterative Gaussian Estimation — arXiv:2611.00259},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00259},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_676776fa6d79; sha256 db0f1688e0367a94d25d1f8472ce09d746ccac4e4c0676a2fbef31c9f339e2e3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00259. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large generative models are commonly trained on data collected from the open web, which may contain data produced by previous generations of the same model. Previous work has shown that recursively training on model-generated data can progressively degrade model performance, a phenomenon known as model collapse. Yet, generative models continue to improve dramatically over time. Existing theory suggests that this is enabled by the availability of new real data at each iteration, kept separate from generated data. However, exact separation is not realistic, and uncurated data is still commonly used. We bridge this gap by considering a more realistic setup in which real and generated data are not handled differently during training. Specifically, we present a comprehensive theoretical study of iterative estimation of the mean and covariance of a multivariate Gaussian distribution. At each iteration, samples are drawn from a mixture of the real data distribution and the estimated distribution from the previous iteration, but are used indistinguishably. We provide a single equivalent condition for consistency of the mean and covariance estimators in Euclidean norms, and convergence to zero of the expected squared 2-Wasserstein distance between the estimated and true distributions. Using this result, we show that consistency holds, i.e., model collapse is avoided, for any fixed real data fraction above zero, and even for a fraction that vanishes across iterations, provided it does not vanish too quickly. However, more real data (or data curation) remains important at practical finite times: the convergence rate of the mean estimator deteriorates as the real data fraction approaches zero. We support our results with an empirical investigation of the theoretical setup, and with experiments on iterative training of a diffusion model on CIFAR-10 that correlate with our theory. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG); Statistics Theory (math.ST). Pagina del lavoro: https://arxiv.org/abs/2611.00259 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:12:54.685Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).