Metraipolis › archive › dc_018a1f93cedd
Entwine: Coordinating Tiled Computation and Fine-Grained Communication across GPUs — arXiv:2609.11562
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11562
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T05:44:51.754Z
- SHA-256 of the archived text
0511965e566b3f1e68990fcbdfbe6ca09d4442b4cce260e725db79d1e5472c89- Stable URL of this page
https://metraipolis.com/documento/dc_018a1f93cedd
How to cite
Entwine: Coordinating Tiled Computation and Fine-Grained Communication across GPUs — arXiv:2609.11562 — arXiv (en), https://arxiv.org/abs/2609.11562 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_018a1f93cedd (Metraipolis archive dc_018a1f93cedd, sha256 0511965e566b3f1e68990fcbdfbe6ca09d4442b4cce260e725db79d1e5472c89)
BibTeX
@misc{dc018a1f93cedd,
title = {Entwine: Coordinating Tiled Computation and Fine-Grained Communication across GPUs — arXiv:2609.11562},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11562},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_018a1f93cedd; sha256 0511965e566b3f1e68990fcbdfbe6ca09d4442b4cce260e725db79d1e5472c89},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11562. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern high-performance GPU computations partition tensors into tiles to exploit data reuse and parallelism. Individual tile computations complete earlier than the full tensor computation, creating opportunities to overlap computation and communication. However, a mismatch between computation and communication progress can limit these opportunities. Communication stalls when no data is ready, and may lag when data arrives in bursts. Communication can also slow computation by consuming shared resources, offsetting the benefits of overlap. We present Entwine, which coordinates tile computation order, fine-grained communication, and SM resource allocation to minimize overall completion time. Entwine reorders tile computation to produce data for communication at a more regular pace. Entwine couples this schedule with fine-grained SM-based communication to process tile results with low latency and low overhead. Since the communication kernel also consumes SM resources, Entwine coordinates their allocation to balance communication progress against computation slowdown. Across representative tensor-parallel LLM workloads, Entwine achieves a geomean speedup of 1.232x (up to 1.433x) over cuBLAS+NCCL, and outperforms state-of-the-art overlap baselines by 3.1-9.8% in geomean. We will open-source our implementation upon publication. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-11 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Hardware Architecture (cs.AR). Pagina del lavoro: https://arxiv.org/abs/2609.11562 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T05:26:44.599Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).