Metraipolis › archive › dc_240303fd259e

Heddle: Learning Structural Templates for Parallelism Planning on Heterogeneous GPU Clusters — arXiv:2609.34244

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34244
License (as archived)
CC0-1.0
Archived
2026-10-07T22:12:11.069Z
SHA-256 of the archived text
a65d197e5c275119deef347496db85a8a0557727f0a8831efef5e3b964423c7e
Stable URL of this page
https://metraipolis.com/documento/dc_240303fd259e

How to cite

Heddle: Learning Structural Templates for Parallelism Planning on Heterogeneous GPU Clusters — arXiv:2609.34244 — arXiv (en), https://arxiv.org/abs/2609.34244 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_240303fd259e (Metraipolis archive dc_240303fd259e, sha256 a65d197e5c275119deef347496db85a8a0557727f0a8831efef5e3b964423c7e)
BibTeX
@misc{dc240303fd259e,
  title = {Heddle: Learning Structural Templates for Parallelism Planning on Heterogeneous GPU Clusters — arXiv:2609.34244},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34244},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_240303fd259e; sha256 a65d197e5c275119deef347496db85a8a0557727f0a8831efef5e3b964423c7e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34244. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Training large machine learning models on shared GPU infrastructures faces two challenges: (1) GPU availability shifts dynamically with varying resource demands from tenants, and (2) hardware heterogeneity accumulates as datacenters continuously adopt new GPU generations. Due to the vast search space induced by heterogeneous GPU types and node sizes, training planners must prune it aggressively to remain tractable, yet must also derive high-throughput plans promptly as cluster configurations change. Heddle achieves this goal through a learning-based planner that reduces the full planning problem to a search over pipeline structures. Heddle encapsulates planning decisions in a structural template and learns to construct plans from templates over diverse cluster configurations offline. This design is effective because structural decisions constitute the performancecritical core of a parallelism plan, while the rest follows by rule or from a small priced candidate set once the plan structure is fixed. Evaluation shows that Heddle matches or exceeds the best plan found by five existing planners across clusters with varying GPU types and node sizes for three models of different sizes by up to 84.5% in throughput on dense models and 4.6x on MoE models.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-29 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.34244 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:53:13.701Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).