Metraipolis › archive › dc_c8cc2ea537a4

CHARM+: Cross-Hardware Attention with Re-Merge Multistream Mechanism — arXiv:2611.05906

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.05906
License (as archived): CC0-1.0
Archived: 2026-11-05T18:16:50.676Z
SHA-256 of the archived text: 945c3535c59820d6bda059ba0a2e952696a3968a94715b27f8a9f16f988c110a
Stable URL of this page: https://metraipolis.com/documento/dc_c8cc2ea537a4

How to cite

CHARM+: Cross-Hardware Attention with Re-Merge Multistream Mechanism — arXiv:2611.05906 — arXiv (en), https://arxiv.org/abs/2611.05906 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c8cc2ea537a4 (Metraipolis archive dc_c8cc2ea537a4, sha256 945c3535c59820d6bda059ba0a2e952696a3968a94715b27f8a9f16f988c110a)

BibTeX

@misc{dcc8cc2ea537a4,
  title = {CHARM+: Cross-Hardware Attention with Re-Merge Multistream Mechanism — arXiv:2611.05906},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05906},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c8cc2ea537a4; sha256 945c3535c59820d6bda059ba0a2e952696a3968a94715b27f8a9f16f988c110a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05906. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As sequence lengths continue to grow, distributed attention based on sequence parallel (e.g., Ring-Attention) has become a mainstream solution for large language models (LLMs). Nevertheless, inherent communication dependencies limit computation-communication overlap, reducing GPU utilization. Moreover, in cross-hardware (NUMA or node) architectures, due to imbalanced communication bandwidth, frequent communication synchronization will limit overall performance. Based on these insights, we propose CHARM+, Cross-Hardware Attention with Re-Merge Multistream Mechanism. We reorganize sequential communications into several levels and achieve intra-level parallelism and computation overlap. Furthermore, we decouple cross-hardware communication from intra-hardware communication, eliminating synchronization overhead caused by bandwidth imbalance. Forward and backward algorithms were evaluated on multiple hardware architectures and consistently outperformed state-of-the-art methods. Compared to Ring-Attention, our forward and backward algorithms achieve average speedups of 1.3x and 1.7x on a dual-NUMA PCIe architecture, and 2.8x and 2.9x on a dual-node NVLink architecture, respectively.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2611.05906 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:12:49.002Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).