Metraipolis › archive › dc_5f0fca107bf0
Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA — arXiv:2609.04261
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04261
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:59:35.276Z
- SHA-256 of the archived text
c710661dd4823c5316c072631102fed69c90e0ce7c96843bdce2f33e44a91efc- Stable URL of this page
https://metraipolis.com/documento/dc_5f0fca107bf0
How to cite
Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA — arXiv:2609.04261 — arXiv (en), https://arxiv.org/abs/2609.04261 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5f0fca107bf0 (Metraipolis archive dc_5f0fca107bf0, sha256 c710661dd4823c5316c072631102fed69c90e0ce7c96843bdce2f33e44a91efc)
BibTeX
@misc{dc5f0fca107bf0,
title = {Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA — arXiv:2609.04261},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04261},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5f0fca107bf0; sha256 c710661dd4823c5316c072631102fed69c90e0ce7c96843bdce2f33e44a91efc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04261. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Self-supervised pretraining has transformed language and vision, but its value for molecular graph neural networks remains contested. We ask whether pretraining on a large unlabelled corpus improves molecular property prediction. We adapt LeJEPA, a predictor-free joint-embedding predictive architecture regularised by Sketched Isotropic Gaussian Regularisation (SIGReg), to molecular graphs, evaluating GPS and Chemprop-style D-MPNN encoders on the Wong et al. [1] antibiotic-activity dataset and ogbg-molhiv using a multi-seed, bootstrap-based protocol. Pretraining improves learned representations but does not robustly improve finetuning. A frozen probe on pretrained embeddings exceeds random initialisation on both tasks (ogbg-molhiv ROC-AUC 0.788 vs 0.665; +0.123), reaching the published self-supervised band, but this does not translate into finetuning gains. On the antibiotic scaffold split, a canonical partition is significant (delta AUPRC +0.041, p = 0.010), but the effect vanishes across five partitions (pooled +0.013, p = 0.095). Finetuning is null on the random split, ogbg-molhiv, and D-MPNN. The representational edge is nevertheless recoverable. Embeddings saturate at ~16-32 effective dimensions, whereas Morgan fingerprints improve to 1024 bits. At matched dimensionality, fingerprints lead validation (0.799 vs 0.782 at 128 dimensions) but trail shifted test scaffolds (0.759 vs 0.788). Truncating embeddings and combining them with a 1024-bit Morgan fingerprint raises ogbg-molhiv ROC-AUC from 0.805 to 0.832 (delta +0.027; 95% CI [+0.003, +0.054]; p = 0.014); an untrained encoder gains nothing (delta -0.003). Thus, pretraining supplies complementary information best realised through feature-level combination, while finetuning gains are weak and partition-dependent. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.LG). Categorie arXiv: Quantitative Methods (q-bio.QM) ; Machine Learning (cs.LG); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2609.04261 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T22:31:32.223Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).