Metraipolis › archive › dc_5d141e620d4d

Statistical Analysis of Synthetic Data with Verification: A Linear Regression Framework — arXiv:2610.40254

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.40254
License (as archived)
CC0-1.0
Archived
2026-11-01T02:27:22.422Z
SHA-256 of the archived text
c567c13880934197f7fdff0ae68fee2aa984f5b830378c515912c19bc1481da8
Stable URL of this page
https://metraipolis.com/documento/dc_5d141e620d4d

How to cite

Statistical Analysis of Synthetic Data with Verification: A Linear Regression Framework — arXiv:2610.40254 — arXiv (en), https://arxiv.org/abs/2610.40254 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5d141e620d4d (Metraipolis archive dc_5d141e620d4d, sha256 c567c13880934197f7fdff0ae68fee2aa984f5b830378c515912c19bc1481da8)
BibTeX
@misc{dc5d141e620d4d,
  title = {Statistical Analysis of Synthetic Data with Verification: A Linear Regression Framework — arXiv:2610.40254},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.40254},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5d141e620d4d; sha256 c567c13880934197f7fdff0ae68fee2aa984f5b830378c515912c19bc1481da8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40254. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Synthetic data are widely used for training modern AI systems. However, their statistical understanding remains limited. It is well recognized in statistics that repeated self-training on synthetic data can amplify errors inherited from pre-trained models, leading to model collapse. Verification plays a crucial role in selecting high-quality synthetic data to help prevent model collapse and improve model performance. We study verified self-training in a linear regression framework, where a pre-trained estimator generates synthetic data with injected generation noise and a verifier selects samples according to a verification threshold. Post-training on the accepted samples updates either all coordinates (full fine-tuning) or only an active subset (partial fine-tuning). Our analysis reveals three potentially counterintuitive effects. We establish that tighter verification does not always lead to lower prediction risk. We also find that a suitable level of injected generation noise can reduce prediction risk, while excessive noise can increase it. Moreover, full and partial fine-tuning can respond differently to the same verification rule, since pre-training error in the frozen coordinates affects both sample selection and post-training risk. We analyze these effects by deriving finite sample risk bounds for four separate regimes determined by the relative scales of the verification threshold, generation noise, and pre-training error. The bounds precisely characterize how verification affects prediction risk across these regimes. In particular, we quantify how the generation noise, the effective accepted sample size, and the covariance of the accepted covariates affect prediction risk. The analysis extends to iterative verified full fine-tuning, where sufficiently tight verification reduces the initial error contribution at a geometric rate.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG); Statistics Theory (math.ST).
Pagina del lavoro: https://arxiv.org/abs/2610.40254 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-01T02:12:44.324Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).