Metraipolis › archive › dc_edf3174ffc1b
A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum — arXiv:2609.20497
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.20497
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:48:58.524Z
- SHA-256 of the archived text
da5cb207a6a022527ed9270bf070233d3fa423f4c6a8c304dd4e8a1cda69d997- Stable URL of this page
https://metraipolis.com/documento/dc_edf3174ffc1b
How to cite
A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum — arXiv:2609.20497 — arXiv (en), https://arxiv.org/abs/2609.20497 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_edf3174ffc1b (Metraipolis archive dc_edf3174ffc1b, sha256 da5cb207a6a022527ed9270bf070233d3fa423f4c6a8c304dd4e8a1cda69d997)
BibTeX
@misc{dcedf3174ffc1b,
title = {A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum — arXiv:2609.20497},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.20497},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_edf3174ffc1b; sha256 da5cb207a6a022527ed9270bf070233d3fa423f4c6a8c304dd4e8a1cda69d997},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20497. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce Adaptive Score-based Routing Balancer (ASRB), a dynamic, score-based request routing mechanism for Kubernetes-based service deployments over the computing continuum. ASRB jointly considers infrastructure-level information, response time measurements, and application-level quality indicators, with a particular focus on serving Machine Learning (ML) workloads. For these workloads, ASRB balances requests over service instances deployed in the continuum, following service provider-defined policies encoded as weighted combinations of QoS criteria to flexibly address latency-accuracy trade-offs. To drive routing decisions and swiftly adapt to changes in the operating environment, ASRB monitors a range of runtime metrics across multiple system layers. To deal with the associated monitoring overhead, particularly important for large-scale deployments, it selectively and adaptively controls monitoring intensity without sacrificing on routing quality. ASRB is implemented without requiring any modifications to Kubernetes, making it straightforward to deploy and operate in existing cluster environments. Our testbed experiments demonstrate the versatility of ASRB: When tuned for latency reduction, it achieves at least 10 ms lower mean response time compared with latency-oriented state-of-the-art routing mechanisms, while it achieves higher accuracy when this is prioritized through specific configurations, thus enabling flexible and operator-controllable trade-offs. At the same time, it attains reduced failure rates, higher responsiveness to changes in the operating environment, and up to ~70% less monitoring cost than relevant state-of-the-art solutions, at the potential expense of only a modest latency penalty in some configurations. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-18 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2609.20497 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:24:19.926Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).