Metraipolis › archive › dc_44ecfc177872
State-Aware Activation Recomputation with Replay Contracts and Selective Preservation — arXiv:2611.00590
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00590
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:01:18.866Z
- SHA-256 of the archived text
209e6130222396ce259a5f22fe2b727265ef15d7192da7804dbcde6f24db889f- Stable URL of this page
https://metraipolis.com/documento/dc_44ecfc177872
How to cite
State-Aware Activation Recomputation with Replay Contracts and Selective Preservation — arXiv:2611.00590 — arXiv (en), https://arxiv.org/abs/2611.00590 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_44ecfc177872 (Metraipolis archive dc_44ecfc177872, sha256 209e6130222396ce259a5f22fe2b727265ef15d7192da7804dbcde6f24db889f)
BibTeX
@misc{dc44ecfc177872,
title = {State-Aware Activation Recomputation with Replay Contracts and Selective Preservation — arXiv:2611.00590},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00590},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_44ecfc177872; sha256 209e6130222396ce259a5f22fe2b727265ef15d7192da7804dbcde6f24db889f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00590. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Activation recomputation saves accelerator memory by replaying forward segments during the backward pass, but replay is only correct if it sees the state the original forward pass saw. Mutable buffers, aliased bytes and random-generator state can change in between, so re-executing unchanged code can differentiate a different computation. We make this state explicit. A versioned replay contract specifies what replay must restore, and a joint planner chooses, per segment, between activation retention, whole-buffer snapshots and selective-region preservation, charging each option its measured memory and recovery cost. Independent derivative and byte-level checks validate the contract, and a Torch/JAX cohort retains all 5,900 execution records, including matches, rejections and divergences. On frozen sparse-write H100 workloads, each measured over fifty fresh processes, joint planning reduces paired step time by 18.16% against a correctly costed eager/whole planner at a 44 GiB budget (approximate 97.5% ratio interval $[.8172,.8195]$). At 32 GiB, where the eager/whole planner has no feasible static policy, joint planning is 4.98% faster than fixed regions at 29.94 rather than 26.20 GiB. A complete CIFAR-100 dense-queue application runs seven methods for 100 encoder and 25 probe epochs at six training seeds, with byte-identical test scores within each seed. There, joint planning uses 22.41% more allocated memory than whole snapshots, with a process-time geometric-mean ratio of 1.0080 (seed range $[.9845,1.0191]$). Together, the results map where selective preservation pays off: sparse writes with large activations benefit, while in the dense-queue workload whole snapshots use less allocated memory. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.00590 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:59:45.725Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).