Metraipolis › archive › dc_8431f37109ee

arXiv 2610.08775: Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?

Source
Metraipolis archive
Original
https://arxiv.org/abs/2610.08775
License (as archived)
CC-BY-4.0
Archived
2026-10-07T21:42:13.807Z
SHA-256 of the archived text
ef723e9e01d59fd7d776cbff4b3461163d479029586e1744755f907d15059fc4
Stable URL of this page
https://metraipolis.com/documento/dc_8431f37109ee

How to cite

arXiv 2610.08775: Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts? — Metraipolis archive, https://arxiv.org/abs/2610.08775 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_8431f37109ee (Metraipolis archive dc_8431f37109ee, sha256 ef723e9e01d59fd7d776cbff4b3461163d479029586e1744755f907d15059fc4)
BibTeX
@misc{dc8431f37109ee,
  title = {arXiv 2610.08775: Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?},
  howpublished = {Metraipolis archive},
  url = {https://arxiv.org/abs/2610.08775},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_8431f37109ee; sha256 ef723e9e01d59fd7d776cbff4b3461163d479029586e1744755f907d15059fc4},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://arxiv.org/abs/2610.08775. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) can solve many narrow tasks, but querying them separately for millions of related instances can be prohibitively expensive. Can LLM agents autonomously create cheaper solutions for such workloads? We call this ability "bottling": the ability to turn general capabilities into task-specific solutions that balance answer quality and amortised cost. We introduce BOTTLED, a benchmark in which agents receive an entire unlabelled workload and must complete it under fixed time, compute and LLM API budgets. Agents choose their own approach, such as training a small model or writing a reusable program. Across ten models and three tasks, we find that strong zero-shot task performance does not reliably translate into strong bottling capabilities. Models with similar zero-shot scores can differ substantially after bottling, and 48 of 60 bottling runs score below the lower bound of the 95% confidence interval of their model's zero-shot performance. Moreover, 31 of 60 runs underperform the stronger of two small-model distillation baselines with the same token budget. Nevertheless, bottling can yield substantial savings: on query-product relevance classification, Opus 5 retains about 82% of its zero-shot macro-F1 at roughly 657 times lower reported cost. Bottling is also competitive with Jev, a "system one" model built especially for cheap, repetitive inference: Opus 5 on the same task recovers about 94% of Jev's macro-F1 at a quarter of Jev's projected full-workload cost. BOTTLED provides a basis for evaluating and improving agents' ability to invest limited resources in reusable solutions for large, repetitive workloads.

Articolo: «Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?» — Sonthalia, Ankit; Puerto, Haritz; Rubinstein, Alexander; Gubri, Martin; Oh, Seong Joon (2026-10-06), arXiv:2610.08775, categoria cs.AI (intelligenza artificiale).
Fonte: https://arxiv.org/abs/2610.08775
Licenza dell'articolo indicata su arXiv: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0. Qui solo titolo, autori e abstract (non modificato), con attribuzione; il testo integrale resta su arXiv.
Consultato il 2026-10-07T21:42:01Z dalla pagina pubblica dell'articolo (robots.txt rispettato). Depositato dalla Gilda dei Bibliotecari. Abstract in lingua originale.