Metraipolis › archive › dc_736a99f25cf7

Procedural Pretraining for Molecular Property Prediction — arXiv:2609.17831

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.17831
License (as archived)
CC0-1.0
Archived
2026-10-15T15:04:27.417Z
SHA-256 of the archived text
56e1130825244ec11db2d8678f9a3a1dba8d8c498ace4dd896a313a217bda48d
Stable URL of this page
https://metraipolis.com/documento/dc_736a99f25cf7

How to cite

Procedural Pretraining for Molecular Property Prediction — arXiv:2609.17831 — arXiv (en), https://arxiv.org/abs/2609.17831 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_736a99f25cf7 (Metraipolis archive dc_736a99f25cf7, sha256 56e1130825244ec11db2d8678f9a3a1dba8d8c498ace4dd896a313a217bda48d)
BibTeX
@misc{dc736a99f25cf7,
  title = {Procedural Pretraining for Molecular Property Prediction — arXiv:2609.17831},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17831},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_736a99f25cf7; sha256 56e1130825244ec11db2d8678f9a3a1dba8d8c498ace4dd896a313a217bda48d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17831. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Molecular property prediction is often limited by the small size of labeled downstream datasets, motivating pretraining on large corpora of unlabeled molecules. In this work, we ask whether useful inductive biases can instead be learned from abstract, procedurally generated data before a model sees any molecular data. We introduce a three-stage training pipeline consisting of procedural pretraining, molecular pretraining on SMILES, and downstream fine-tuning, and evaluate several procedural tasks spanning sequence structure, cellular automata, and graph reasoning. We find that procedural pretraining can improve molecular property prediction even after subsequent molecular pretraining: on Lipophilicity, \textsc{Reverse} reduces test error by 4.8\%. For context, the magnitude of this improvement is roughly 90\% of the performance difference between our 250K-molecule baseline and the publicly released MoLFormer checkpoint pretrained on approximately 100M molecules. Our analysis shows that the benefit is strongest under downstream data scarcity, depends on the structure of the procedural data rather than only surface-level statistics, and does not increase monotonically with additional procedural training. Instead, transfer typically peaks at an intermediate procedural budget and deteriorates as the model approaches convergence on the procedural task. We further find that, for several tasks, much of the transferable information is localized in the attention layers, while feed-forward layers can contribute to over-specialization. These results show that procedural data can provide transferable structure for molecular learning and offer a complementary route to improving performance when labeled molecular data are limited.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.17831 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.073Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).