Metraipolis › archive › dc_613a765d6361

The Composition Gap in Dataset Distillation — arXiv:2609.34343

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34343
License (as archived)
CC0-1.0
Archived
2026-10-07T22:38:10.796Z
SHA-256 of the archived text
99fc949314534d98601aad3d4a72b815f48a6560c2616fe838165a64da06aa1c
Stable URL of this page
https://metraipolis.com/documento/dc_613a765d6361

How to cite

The Composition Gap in Dataset Distillation — arXiv:2609.34343 — arXiv (en), https://arxiv.org/abs/2609.34343 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_613a765d6361 (Metraipolis archive dc_613a765d6361, sha256 99fc949314534d98601aad3d4a72b815f48a6560c2616fe838165a64da06aa1c)
BibTeX
@misc{dc613a765d6361,
  title = {The Composition Gap in Dataset Distillation — arXiv:2609.34343},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34343},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_613a765d6361; sha256 99fc949314534d98601aad3d4a72b815f48a6560c2616fe838165a64da06aa1c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34343. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dataset distillation compresses a training set into a small synthetic set, usually evaluated one at a time. In federated and data-governance settings, several parties distill their own data and a user trains on their union. We ask whether the union of separately distilled sets reproduces training on the union of the real data composability and show that it can fail even when every source is distilled exactly and the total budget admits an exact joint distillate. Compressing a training trajectory into fewer steps transforms the source statistics nonlinearly, so averaging compressed sources differs from compressing their average. For quadratic objectives we derive the exact composition error for two-to-one step compression in terms of the source-Hessian variance and the linear terms of the losses, and on a smooth network at small step sizes this prediction captures the local endpoint discrepancy in magnitude and direction. For learned synthetic sets, however, the composed error decomposes exactly into this local discrepancy and an aggregate source residual. Under endpoint matching the residual exceeds the structural term by more than an order of magnitude, and under distribution matching the two terms partly cancel. Joint distillation also retains an accuracy advantage when both sets are distilled from the same dataset, where the local discrepancy is exactly zero. Training fidelity and downstream accuracy are therefore distinct requirements, neither established by evaluating each set on its own.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.34343 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:28.160Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).