Metraipolis › archive › dc_981532bbf2a1

Refresh or Realize? Compute Allocation in Drifting Models — arXiv:2609.32298

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32298
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:27.984Z
SHA-256 of the archived text
c2fbea0799a67027ae144405c08180155d2b1d1b507e908c185c04da3677663c
Stable URL of this page
https://metraipolis.com/documento/dc_981532bbf2a1

How to cite

Refresh or Realize? Compute Allocation in Drifting Models — arXiv:2609.32298 — arXiv (en), https://arxiv.org/abs/2609.32298 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_981532bbf2a1 (Metraipolis archive dc_981532bbf2a1, sha256 c2fbea0799a67027ae144405c08180155d2b1d1b507e908c185c04da3677663c)
BibTeX
@misc{dc981532bbf2a1,
  title = {Refresh or Realize? Compute Allocation in Drifting Models — arXiv:2609.32298},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32298},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_981532bbf2a1; sha256 c2fbea0799a67027ae144405c08180155d2b1d1b507e908c185c04da3677663c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32298. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Drifting Models train a one-step generator by recomputing a finite-sample drift field at every iteration and taking an optimizer step toward the drifted target. The field says how generated samples should move, but the step is taken in parameters shared by all samples, so the motion the network actually makes need not match the motion it was given. This leaves a basic training question open: should extra compute go into fitting the current target more closely, or into recomputing the field? We study it on ImageNet 256x256. Holding the target fixed for k optimizer steps and measuring the realized displacement, we find that deeper fitting does bring the network closer to the frozen target, and that the number of steps needed before it makes any net progress drops from about sixteen early in training to one later on. When the extra steps come for free, k=2 also lowers FID. Once they are paid for, the result flips: at approximately matched measured wall-clock, spending the budget on fresh fields gives lower FID than deeper fitting, on both training seeds. The target itself shows why a fresh field is worth so much. Redrawing the finite support rotates its direction far more than a parameter update does (cosine ~0.3-0.6 against ~0.95), and a correction that is optimal in field space is not reliably better in FID than a parameter-free one. For Drifting, fitting each target well and spending compute well are different goals.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32298 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.647Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).