Metraipolis › archive › dc_e0ebfed36b17
Mask-Aware Execution for Efficient JEPA Training — arXiv:2609.22674
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22674
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:52:58.718Z
- SHA-256 of the archived text
cf62723cb60907645e3ae088e6505a39076b2170ff1ac9160a9b785255222ef1- Stable URL of this page
https://metraipolis.com/documento/dc_e0ebfed36b17
How to cite
Mask-Aware Execution for Efficient JEPA Training — arXiv:2609.22674 — arXiv (en), https://arxiv.org/abs/2609.22674 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e0ebfed36b17 (Metraipolis archive dc_e0ebfed36b17, sha256 cf62723cb60907645e3ae088e6505a39076b2170ff1ac9160a9b785255222ef1)
BibTeX
@misc{dce0ebfed36b17,
title = {Mask-Aware Execution for Efficient JEPA Training — arXiv:2609.22674},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22674},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e0ebfed36b17; sha256 cf62723cb60907645e3ae088e6505a39076b2170ff1ac9160a9b785255222ef1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22674. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Joint Embedding Predictive Architectures (JEPAs) are becoming a core representation-learning primitive and a building block for latent world models across vision, video, audio, brain dynamics, and time series. Despite (potential of) wide deployment, current JEPA training pipelines are inefficient: each input is executed through multiple mask-specific branches, with redundant target-side work, and memory-bound token routing. These costs grow with the number of masks and limit GPU efficiency. We present M-JEPA, a mask-aware execution architecture that restructures JEPA training without changing the learning objective. M-JEPA separates mask-independent computation from mask-dependent routing, enabling shared context encoder execution, fused token routing and slicing with backward support, sparse target encoder execution over the union of target tokens, and masked patch embedding for sparse inputs. The resulting pipeline preserves training semantics while reducing computation, memory traffic, and synchronization overhead. We implement M-JEPA for five JEPA variants and evaluate it on NVIDIA A100 GPUs. Compared against the state-of-the-art baselines, M-JEPA achieves up to 1.7x end-to-end training speedup for 2-10 masks. Separately, with masked patch embedding, 4.75x patch-embedding speedup at high sparsity. These results show that execution restructuring, rather than changes to the JEPA objective, is a key lever for efficient JEPA training. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-22 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.22674 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:50:13.516Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).