Metraipolis › archive › dc_be31ec12834d

Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty — arXiv:2610.41745

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41745
License (as archived)
CC0-1.0
Archived
2026-11-02T04:52:25.794Z
SHA-256 of the archived text
e14741fa6ee2587146d26b2999f9f040b1e07d18bc72fafb8cafd299b1b1c2ac
Stable URL of this page
https://metraipolis.com/documento/dc_be31ec12834d

How to cite

Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty — arXiv:2610.41745 — arXiv (en), https://arxiv.org/abs/2610.41745 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_be31ec12834d (Metraipolis archive dc_be31ec12834d, sha256 e14741fa6ee2587146d26b2999f9f040b1e07d18bc72fafb8cafd299b1b1c2ac)
BibTeX
@misc{dcbe31ec12834d,
  title = {Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty — arXiv:2610.41745},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41745},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_be31ec12834d; sha256 e14741fa6ee2587146d26b2999f9f040b1e07d18bc72fafb8cafd299b1b1c2ac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41745. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are used in many domains, but molecular property prediction with them is still little explored. In this work we evaluate the capability of 7 frontier LLMs to predict molecular properties together with a self-reported uncertainty from in-context examples, and compare them with classical baselines trained on the same molecules. We use 4 datasets and screen each for contamination of the training data of the LLMs: three are clean, and the fourth, ESOL, is memorised by most of the models and serves as a comparison. On the clean datasets every LLM improves with the context, at 30 examples the most accurate one beats every baseline. The requested reasoning effort changes accuracy little. The self-reported uncertainty ranks the prediction errors weakly, with Spearman rank correlations of up to 0.46, about the level reached by the uncertainties of the baselines. Combining uncertainty and prediction, on lipophilicity the most certain half of the best LLM is the most accurate at small context sizes; on BACE and antiviral, and on lipophilicity at large context sizes, the best LLMs are on par with the best baseline. On the contaminated ESOL dataset the LLMs show no improvement with the context size, as expected for a memorised dataset, and their stated uncertainty is smaller on the molecules whose value they recall.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.41745 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T04:45:12.907Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).