Metraipolis › archive › dc_734efa347da4

SCOUT: Symmetric Consensus Outlier Detection for Failure Localization in LLM Pre-Training — arXiv:2608.11034

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.11034
License (as archived): CC0-1.0
Archived: 2026-10-19T06:14:36.017Z
SHA-256 of the archived text: 05ab2d87353b5be689a2f6fda2ed6be20ad646ee7de61090ab8ab86500276722
Stable URL of this page: https://metraipolis.com/documento/dc_734efa347da4

How to cite

SCOUT: Symmetric Consensus Outlier Detection for Failure Localization in LLM Pre-Training — arXiv:2608.11034 — arXiv (en), https://arxiv.org/abs/2608.11034 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_734efa347da4 (Metraipolis archive dc_734efa347da4, sha256 05ab2d87353b5be689a2f6fda2ed6be20ad646ee7de61090ab8ab86500276722)

BibTeX

@misc{dc734efa347da4,
  title = {SCOUT: Symmetric Consensus Outlier Detection for Failure Localization in LLM Pre-Training — arXiv:2608.11034},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11034},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_734efa347da4; sha256 05ab2d87353b5be689a2f6fda2ed6be20ad646ee7de61090ab8ab86500276722},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11034. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In LLM pre-training, synchronization propagates rank-local stalls, slowdowns, and numerical errors into job-wide symptoms, obscuring their origin. Existing diagnosis often relies on in-process monitors that cannot report after the trainer blocks or terminates, or on post-mortem logs that preserve only synchronized symptoms; offline health tests lose the workload and operating conditions that triggered the failure. We present SCOUT, a unified runtime failure-localization framework built on one design principle: identify outliers through strict-majority consensus among equivalent replicas. SCOUT aligns replica progress, timing, and numerical evidence, then uses its Consensus Collective Communication (C3) abstraction to identify ranks whose compact signatures disagree with their peers. An out-of-band CPU observer remains responsive when training hangs, whereas in-situ replay exercises recurring stragglers and silent data corruption (SDC) beside the live job with its model state, kernels, allocations, communication path, and thermal and memory pressure present. Collective fingerprints expose rank-local protocol divergence. Clean replay coverage certifies checkpoint numerical integrity, preventing recovery from selecting state corrupted by SDC. SCOUT integrates with PyTorch, TorchTitan, Megatron-Core, and DeepSpeed without training-loop or framework-source modifications. SCOUT is open source at this https URL .

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-12 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.11034 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:09:08.093Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).