Metraipolis › archive › dc_e34249109dfd

UMLoad: Fast On-Demand Model Loading on Unified Memory Architecture — arXiv:2611.04253

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.04253
License (as archived): CC0-1.0
Archived: 2026-11-04T21:46:12.298Z
SHA-256 of the archived text: 56e82d1023439ac8a3bd91e14b4602e1649bc8eb363ba1cd3fc7378bbd260790
Stable URL of this page: https://metraipolis.com/documento/dc_e34249109dfd

How to cite

UMLoad: Fast On-Demand Model Loading on Unified Memory Architecture — arXiv:2611.04253 — arXiv (en), https://arxiv.org/abs/2611.04253 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e34249109dfd (Metraipolis archive dc_e34249109dfd, sha256 56e82d1023439ac8a3bd91e14b4602e1649bc8eb363ba1cd3fc7378bbd260790)

BibTeX

@misc{dce34249109dfd,
  title = {UMLoad: Fast On-Demand Model Loading on Unified Memory Architecture — arXiv:2611.04253},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04253},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e34249109dfd; sha256 56e82d1023439ac8a3bd91e14b4602e1649bc8eb363ba1cd3fc7378bbd260790},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04253. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-demand model loading dominates response latency for interactive on-device AI, yet unified memory architectures (UMAs), now common on edge devices, should make it cheap by placing weights in memory the GPU already sees. However, frameworks still use the loading flow built for discrete GPUs, leaving significant inefficiencies. This paper presents UMLoad, a system-level model-loading accelerator that decouples a framework's logical allocations from their physical page backing. UMLoad uses on-demand reconstruction to eliminate the weight copy while preserving copy independence, and GPU-assisted page zeroing to accelerate runtime allocation while preserving its security guarantee. Implemented on Intel, AMD, and NVIDIA UMA platforms and evaluated with this http URL , PyTorch, and vLLM across six models, UMLoad reduces average loading time by 36.6%-71.9% over vanilla and 19.9%-62.7% over pipelined loaders, and cuts mean LLM-routing time to first token (TTFT) by 51.3%-59.3% and model-backed tool completion latency by 10.2%-31.6%.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno).
Categorie arXiv: Operating Systems (cs.OS) ; Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2611.04253 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:36:34.398Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).