Metraipolis › archive › dc_3a09fd425b15
State-Aware Activation Recomputation with Replay Contracts and Selective Preservation — arXiv:2611.00590
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00590
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:30:55.283Z
- SHA-256 of the archived text
da35912c2daf021cd8405ffa3e6b77153536b0e9d490d03adedf12f5c7fb130a- Stable URL of this page
https://metraipolis.com/documento/dc_3a09fd425b15
How to cite
State-Aware Activation Recomputation with Replay Contracts and Selective Preservation — arXiv:2611.00590 — arXiv (en), https://arxiv.org/abs/2611.00590 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3a09fd425b15 (Metraipolis archive dc_3a09fd425b15, sha256 da35912c2daf021cd8405ffa3e6b77153536b0e9d490d03adedf12f5c7fb130a)
BibTeX
@misc{dc3a09fd425b15,
title = {State-Aware Activation Recomputation with Replay Contracts and Selective Preservation — arXiv:2611.00590},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00590},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3a09fd425b15; sha256 da35912c2daf021cd8405ffa3e6b77153536b0e9d490d03adedf12f5c7fb130a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00590. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Activation recomputation saves accelerator memory by replaying forward segments during the backward pass, but replay is only correct if it sees the state the original forward pass saw. Mutable buffers, aliased bytes and random-generator state can change in between, so re-executing unchanged code can differentiate a different computation. We make this state explicit. A versioned replay contract specifies what replay must restore, and a joint planner chooses, per segment, between activation retention, whole-buffer snapshots and selective-region preservation, charging each option its measured memory and recovery cost. Independent derivative and byte-level checks validate the contract, and a Torch/JAX cohort retains all 5,900 execution records, including matches, rejections and divergences. On frozen sparse-write H100 workloads, each measured over fifty fresh processes, joint planning reduces paired step time by 18.16% against a correctly costed eager/whole planner at a 44 GiB budget (approximate 97.5% ratio interval $[.8172,.8195]$). At 32 GiB, where the eager/whole planner has no feasible static policy, joint planning is 4.98% faster than fixed regions at 29.94 rather than 26.20 GiB. A complete CIFAR-100 dense-queue application runs seven methods for 100 encoder and 25 probe epochs at six training seeds, with byte-identical test scores within each seed. There, joint planning uses 22.41% more allocated memory than whole snapshots, with a process-time geometric-mean ratio of 1.0080 (seed range $[.9845,1.0191]$). Together, the results map where selective preservation pays off: sparse writes with large activations benefit, while in the dense-queue workload whole snapshots use less allocated memory. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.00590 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:29:48.460Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).