Metraipolis › archive › dc_7bae285b9a64

Bounding Queue Waits for Fair Scheduling of Bursty HPC Workloads via Hierarchical Algorithms — arXiv:2610.32027

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32027
License (as archived)
CC0-1.0
Archived
2026-10-26T19:54:37.411Z
SHA-256 of the archived text
d9237809fa1644fe1207c2f291e938adb08960793955046ccfe02f8052c896c7
Stable URL of this page
https://metraipolis.com/documento/dc_7bae285b9a64

How to cite

Bounding Queue Waits for Fair Scheduling of Bursty HPC Workloads via Hierarchical Algorithms — arXiv:2610.32027 — arXiv (en), https://arxiv.org/abs/2610.32027 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7bae285b9a64 (Metraipolis archive dc_7bae285b9a64, sha256 d9237809fa1644fe1207c2f291e938adb08960793955046ccfe02f8052c896c7)
BibTeX
@misc{dc7bae285b9a64,
  title = {Bounding Queue Waits for Fair Scheduling of Bursty HPC Workloads via Hierarchical Algorithms — arXiv:2610.32027},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32027},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7bae285b9a64; sha256 d9237809fa1644fe1207c2f291e938adb08960793955046ccfe02f8052c896c7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32027. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

HPC clusters are shared among many projects, each holding an allocation of node-hours, and ensuring fairness is becoming harder. Foundation models workflows and automated pipelines have made job submissions bursty and concentrated among a few heavy users, so occasional users absorb queueing delays created by frequent ones, with no indication of how long their jobs will wait. Existing schedulers either forget past usage, protect only the job at the head of the queue, or assume that running jobs can be preempted. We propose HyDRA, a Hybrid Deadline and Ranking-Aware scheduler that enforces fairness among users while bounding the queue wait of every job. HyDRA assigns each job a deadline on arrival based on its size, the priorities of its project and user, the current backlog, and how far its project and user are from their entitled share. It ranks waiting jobs by a priority score that grows with waiting time, serves jobs close to their deadlines first, and backfills a job only when no deadline, or fairness limit is violated. We evaluate HyDRA on workloads derived from a year of ALCF Polaris job traces, against conventional schedulers and variants inspired by PCS, Themis, and the PBS job-sort formula used at ALCF. HyDRA achieves $2\times$ the project-level Jain's fairness of the best baseline, starts 47\% of jobs by their deadlines compared with at most 36\% for the baselines even for largest workloads, and lowers the 95th-\%ile bounded slowdown by 4.2$\times$ relative to the next best baseline, at a moderate cost in utilization.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.32027 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:46:03.291Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).