Metraipolis › archive › dc_505eb85b2ecd

Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters — arXiv:2609.04417

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.04417
License (as archived)
CC0-1.0
Archived
2026-10-16T22:41:20.889Z
SHA-256 of the archived text
f20e031fb8bd2b11f37986eac416dfc8e79bedaf42725610092ce8b1755abb36
Stable URL of this page
https://metraipolis.com/documento/dc_505eb85b2ecd

How to cite

Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters — arXiv:2609.04417 — arXiv (en), https://arxiv.org/abs/2609.04417 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_505eb85b2ecd (Metraipolis archive dc_505eb85b2ecd, sha256 f20e031fb8bd2b11f37986eac416dfc8e79bedaf42725610092ce8b1755abb36)
BibTeX
@misc{dc505eb85b2ecd,
  title = {Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters — arXiv:2609.04417},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.04417},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_505eb85b2ecd; sha256 f20e031fb8bd2b11f37986eac416dfc8e79bedaf42725610092ce8b1755abb36},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04417. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Distributed AI training involves recurring rounds of data exchange between multiple pairs of GPU nodes. Slowdown in even one flow due to congestion can cause the entire communication round to slowdown. Current approaches for evading congestion in AI clusters assume global control over the entire workload (e.g. coordinating the schedule of all jobs) or assume infrastructural support (e.g. adaptive routing in switches). They are thus ill-suited in a shared cloud setting where AI jobs belonging to one user can face external congestion from other users' jobs or background traffic beyond its own control. In this paper, we build a system, REACT, that tunes the recurring pattern of data exchange between GPU nodes (known as communication collectives) in response to congestion. REACT works at the application (communication library) layer, where it detects congestion at runtime using readily available flow stats, and tunes the collective pattern to alleviate congestion - changing the set of incident flows while retaining the semantics of information exchange (e.g. selecting which node aggregates data in an AllReduce tree). REACT requires no explicit support from the underlying network infrastructure and can be unilaterally deployed by individual users in a shared cloud setting. We prototype REACT as a shim layer over NCCL, and evaluate it on a shared academic GPU cluster - enabling REACT improves communication performance (algorithm bandwidth) by 13%-38% under network congestion. Our simulations across a range of congestion scenarios further reveal up to 75% performance improvement, highlighting the effectiveness of our approach.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-07 in cs.DC).
Categorie arXiv: Networking and Internet Architecture (cs.NI) ; Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.04417 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:26:34.123Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).