Metraipolis › archive › dc_281af105cb09

Finite-Horizon Input-Output Dynamics of Minibatch Perturbations in AdamW — arXiv:2608.19762

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.19762
License (as archived)
CC0-1.0
Archived
2026-10-18T11:48:15.501Z
SHA-256 of the archived text
476192ea547a9c8bb1435745caf5d6eec72dd32075f82f3e691068397afb9235
Stable URL of this page
https://metraipolis.com/documento/dc_281af105cb09

How to cite

Finite-Horizon Input-Output Dynamics of Minibatch Perturbations in AdamW — arXiv:2608.19762 — arXiv (en), https://arxiv.org/abs/2608.19762 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_281af105cb09 (Metraipolis archive dc_281af105cb09, sha256 476192ea547a9c8bb1435745caf5d6eec72dd32075f82f3e691068397afb9235)
BibTeX
@misc{dc281af105cb09,
  title = {Finite-Horizon Input-Output Dynamics of Minibatch Perturbations in AdamW — arXiv:2608.19762},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.19762},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_281af105cb09; sha256 476192ea547a9c8bb1435745caf5d6eec72dd32075f82f3e691068397afb9235},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.19762. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A minibatch can influence training beyond the update at which it is observed because AdamW stores past gradient information in its optimizer states. We study this delayed effect through paired trajectories that differ only in one gradient update and share the same subsequent training sequence. We formulate AdamW as a finite-horizon input--state--output (ISO) system whose state contains the model parameters and first- and second-moment estimates. Linearizing the joint dynamics yields a signed response operator that maps a localized gradient perturbation to its future loss effects, revealing how optimizer memory shapes their magnitude, timing, and sign. We further derive an exact multistep error decomposition and establish first-order finite-horizon accuracy under local smoothness and controlled activation switching. Experiments validate the response mechanism and optimizer-state effects, while repeated-future analyses reveal substantial prospective structure in delayed influence that can be partially recovered from ISO approximations. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-21 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2608.19762 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T11:47:43.316Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).