Metraipolis › archive › dc_2fe214fc6753
Harnessing Large Language Models to Compile Task-Relevant Context into Bayesian Optimisation — arXiv:2609.36788
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36788
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T17:29:28.223Z
- SHA-256 of the archived text
e4b4287d0e92e2a9219bf63e6c8528189d26f6c49168397be12d992d2be9b7fc- Stable URL of this page
https://metraipolis.com/documento/dc_2fe214fc6753
How to cite
Harnessing Large Language Models to Compile Task-Relevant Context into Bayesian Optimisation — arXiv:2609.36788 — arXiv (en), https://arxiv.org/abs/2609.36788 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2fe214fc6753 (Metraipolis archive dc_2fe214fc6753, sha256 e4b4287d0e92e2a9219bf63e6c8528189d26f6c49168397be12d992d2be9b7fc)
BibTeX
@misc{dc2fe214fc6753,
title = {Harnessing Large Language Models to Compile Task-Relevant Context into Bayesian Optimisation — arXiv:2609.36788},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36788},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2fe214fc6753; sha256 e4b4287d0e92e2a9219bf63e6c8528189d26f6c49168397be12d992d2be9b7fc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36788. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Incorporating rich task-relevant context, such as domain knowledge and external observations, is a key capability yet remains challenging for Bayesian optimisation (BO). Recently, practitioners have started to use large language models (LLMs) to generate and execute BO programs through coding harnesses. In such emerging practices, the posterior belief is shaped not only by Bayesian inference but also by LLM-generated model and data artefacts, offering a flexible route for task context to enter BO as executable code. To study whether and how LLMs can be harnessed to compile diverse contextual signals for BO, we formulate LLM-compiled BO as generalised-context decision making. We propose HarBO, a BO-specialised harness that compiles generalised context into the core artefacts of standard BO through a validated multi-stage workflow. Our theory analyses the regret under imperfect compilation and the effect of adding new context. Across synthetic functions and real-world benchmarks, we find that LLM harnesses can effectively compile context into standard BO, achieving competitive performance with specialised LLM-embedding-based and direct LLM-in-the-loop BO methods. General coding harnesses can be effective in familiar domains such as hyperparameter optimisation, but fall short in unfamiliar, context-rich domains. Together, these results establish LLM harnesses as a promising, but not automatically reliable, route for making rich task context usable in BO. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.36788 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T16:59:48.236Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).