Metraipolis › archive › dc_89269c0aa5bf
Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty — arXiv:2610.41745
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41745
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T05:40:57.348Z
- SHA-256 of the archived text
40b0c3321d90ad1a9f405dcc34fe51d56ebcef4e46dc023641261bf009433185- Stable URL of this page
https://metraipolis.com/documento/dc_89269c0aa5bf
How to cite
Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty — arXiv:2610.41745 — arXiv (en), https://arxiv.org/abs/2610.41745 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89269c0aa5bf (Metraipolis archive dc_89269c0aa5bf, sha256 40b0c3321d90ad1a9f405dcc34fe51d56ebcef4e46dc023641261bf009433185)
BibTeX
@misc{dc89269c0aa5bf,
title = {Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty — arXiv:2610.41745},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41745},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89269c0aa5bf; sha256 40b0c3321d90ad1a9f405dcc34fe51d56ebcef4e46dc023641261bf009433185},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41745. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) are used in many domains, but molecular property prediction with them is still little explored. In this work we evaluate the capability of 7 frontier LLMs to predict molecular properties together with a self-reported uncertainty from in-context examples, and compare them with classical baselines trained on the same molecules. We use 4 datasets and screen each for contamination of the training data of the LLMs: three are clean, and the fourth, ESOL, is memorised by most of the models and serves as a comparison. On the clean datasets every LLM improves with the context, at 30 examples the most accurate one beats every baseline. The requested reasoning effort changes accuracy little. The self-reported uncertainty ranks the prediction errors weakly, with Spearman rank correlations of up to 0.46, about the level reached by the uncertainties of the baselines. Combining uncertainty and prediction, on lipophilicity the most certain half of the best LLM is the most accurate at small context sizes; on BACE and antiviral, and on lipophilicity at large context sizes, the best LLMs are on par with the best baseline. On the contaminated ESOL dataset the LLMs show no improvement with the context size, as expected for a memorised dataset, and their stated uncertainty is smaller on the molecules whose value they recall. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.41745 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T05:26:25.712Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).