Metraipolis › archive › dc_4448839b2fc5

Peak-Memory Schedules You Can Trust: Exact Realization, Certified Optimality, and the Limits of Recomputation for Inference — arXiv:2610.23328

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.23328
License (as archived)
CC0-1.0
Archived
2026-10-19T08:44:27.125Z
SHA-256 of the archived text
84a307f01110fe48949b075911c7313b2b134e2fca93ce4f6361a677277aa598
Stable URL of this page
https://metraipolis.com/documento/dc_4448839b2fc5

How to cite

Peak-Memory Schedules You Can Trust: Exact Realization, Certified Optimality, and the Limits of Recomputation for Inference — arXiv:2610.23328 — arXiv (en), https://arxiv.org/abs/2610.23328 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4448839b2fc5 (Metraipolis archive dc_4448839b2fc5, sha256 84a307f01110fe48949b075911c7313b2b134e2fca93ce4f6361a677277aa598)
BibTeX
@misc{dc4448839b2fc5,
  title = {Peak-Memory Schedules You Can Trust: Exact Realization, Certified Optimality, and the Limits of Recomputation for Inference — arXiv:2610.23328},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.23328},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4448839b2fc5; sha256 84a307f01110fe48949b075911c7313b2b134e2fca93ce4f6361a677277aa598},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23328. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Real-time inference on small devices needs low activation memory without extra latency, which makes operator order the most valuable memory lever: it is free in time. We show that the usual way of deploying a better order, handing it back to the compiler and regenerating code, is unreliable. In PyTorch Inductor the regenerated program misses its predicted peak in 22 of 54 CPU configurations. It is worse than the stock program in 9 of them and in 19 of 50 on a GPU, and on the GPU it also changes numerical results. We instead profile the generated program, model its memory exactly from its own statements and their read/write effects, and permute those statements. We prove that this realizes the modeled peak whenever allocation does not depend on history. It does so to the byte, with bitwise-identical results, in all our CPU runs. On the GPU we characterize why the caching allocator breaks the condition. To know when to stop, we give forward and reverse "trap" lower bounds, each computable with $O(n^2)$ minimum cuts. They certify the found order optimal for 94% of held-out wrapper graphs and all 16 microcontroller-class models we study, in under a second. So most inference graphs are already scheduled optimally. Branching networks are the exception, where reordering is free and pays, e.g. 10.0% on DeepLabV3. In compiled training we save up to 7.6%. Beyond the certified optimum we study selective recomputation with per-operator caps. It is strongly NP-hard for every fixed cap even at treewidth two, encodes knapsack exactly on series-parallel graphs, and is solvable exactly on bounded treewidth. It is useless on public tiny models whose peak is a single layer, but on architectures that hold a cheap value for a late reuse it pays: on the public ReDimNet speaker networks, whose order is already optimal, it lowers the peak by 5.2-6.1% at under 0.4% extra MACs, with bitwise-identical outputs.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.PF, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Data Structures and Algorithms (cs.DS); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.23328 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:40:09.825Z dalla pagina https://arxiv.org/list/cs.PF/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).