Metraipolis › archive › dc_d5281dacc3bd

BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems — arXiv:2608.02612

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02612
License (as archived)
CC0-1.0
Archived
2026-10-31T12:30:50.301Z
SHA-256 of the archived text
66548885cdf890803e1ac6a02b6d663f993b8b6b8c257606249a4e848632bf85
Stable URL of this page
https://metraipolis.com/documento/dc_d5281dacc3bd

How to cite

BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems — arXiv:2608.02612 — arXiv (en), https://arxiv.org/abs/2608.02612 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d5281dacc3bd (Metraipolis archive dc_d5281dacc3bd, sha256 66548885cdf890803e1ac6a02b6d663f993b8b6b8c257606249a4e848632bf85)
BibTeX
@misc{dcd5281dacc3bd,
  title = {BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems — arXiv:2608.02612},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02612},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d5281dacc3bd; sha256 66548885cdf890803e1ac6a02b6d663f993b8b6b8c257606249a4e848632bf85},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02612. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Formulating an optimization problem strongly affects the quality of the final solution, yet good formulations usually require substantial expertise. Recent studies have therefore examined how to automatically derive optimization problems from natural-language descriptions, but existing benchmarks focus on settings where objectives and constraints can be written explicitly as mathematical expressions. Many practically important problems are naturally treated as black-box optimization (BBO) problems, in which only objective values are observable, and the functional form is unavailable. In BBO, the search space design, a part of the problem formulation, and the selection of the optimization algorithm are crucial for problem-solving. Automating these processes with large language models (LLMs) is a significant challenge. This paper introduces Black-Box Optimization Word Problems (BBOWP), a novel problem setting in which a system must infer both a search space and an optimization algorithm from a natural-language description of a black-box optimization task. To support research on this setting, we establish the BBOWP Benchmark Suite (BBOWP-Bench), a dataset and evaluation framework for BBOWP. Each instance combines a natural-language problem description, an executable evaluation environment, and a human-designed baseline formulation, allowing evaluation of both search-space design and algorithm selection. Using this benchmark, we provide the first evaluation of LLMs and show that current LLMs are capable of selecting suitable algorithms based on the given evaluation budget. However, they sometimes struggle with search space design, particularly in identifying important variables and balancing their ranges when the problem description is less informative or the search space is highly problem-specific. Our code and dataset are available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE).
Pagina del lavoro: https://arxiv.org/abs/2608.02612 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:17:37.715Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).