Metraipolis › archive › dc_02f52accfb7f

Why Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model Serving — arXiv:2611.11739

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.11739
License (as archived)
CC0-1.0
Archived
2026-11-10T18:45:47.351Z
SHA-256 of the archived text
bff271dc4b11340ec90ee3daf5c86be39c9df0ad02fb1236676406b7d0467dbf
Stable URL of this page
https://metraipolis.com/documento/dc_02f52accfb7f

How to cite

Why Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model Serving — arXiv:2611.11739 — arXiv (en), https://arxiv.org/abs/2611.11739 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_02f52accfb7f (Metraipolis archive dc_02f52accfb7f, sha256 bff271dc4b11340ec90ee3daf5c86be39c9df0ad02fb1236676406b7d0467dbf)
BibTeX
@misc{dc02f52accfb7f,
  title = {Why Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model Serving — arXiv:2611.11739},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.11739},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_02f52accfb7f; sha256 bff271dc4b11340ec90ee3daf5c86be39c9df0ad02fb1236676406b7d0467dbf},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11739. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Lazy container-image pulling makes cold starts of autoscaled model-serving fleets on Kubernetes nearly independent of model size, but it moves the model's bytes into a node-level cache that the platform neither bounds nor accounts for. This article establishes, by experiment, two impossibility results for that cache. A monitoring-side remedy cannot catch its failure: a sentinel readiness probe stays green while 272 of 280 files are unreadable. A platform-side remedy cannot contain it: kubelet, given full disk-pressure visibility, frees 0 bytes and evicts 26 bystander pods, because its reclamation actuator is keyed to an accounting that does not know the cache exists. The article then eliminates the failure class inside the component that owns the bytes, with no change to kubelet or the container runtime interface, by two design rules: a byte budget whose pressure signal is keyed to filesystem truth, so that a never-blocking, lossy accounting still enforces a hard bound (held against about 301,000 dropped updates), and pass-through semantics under which a failed cache write never becomes a failed read, with the coupling traced to a single this http URL . Across six pre-specified campaigns on AWS, upstream stargz-snapshotter fails 273 of 280 model files at 90% cache pre-fill; the proposed system reads all 150.3 GB with zero errors at +18.5% completion cost and holds the budget under a live read, co-tenancy, and a three-node fleet. A source audit shows the missing bound generalizes to SOCI and Nydus. The implementation is open source and has been proposed upstream.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Operating Systems (cs.OS); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2611.11739 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:38:07.444Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).