Metraipolis › archive › dc_223bd47bbe72

ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience? — arXiv:2610.07792

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.07792
License (as archived)
CC0-1.0
Archived
2026-10-07T11:21:12.448Z
SHA-256 of the archived text
0add03c74376b11cc161b949033a808befc2e0c6c53f27a51792c802adc91941
Stable URL of this page
https://metraipolis.com/documento/dc_223bd47bbe72

How to cite

ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience? — arXiv:2610.07792 — arXiv (en), https://arxiv.org/abs/2610.07792 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_223bd47bbe72 (Metraipolis archive dc_223bd47bbe72, sha256 0add03c74376b11cc161b949033a808befc2e0c6c53f27a51792c802adc91941)
BibTeX
@misc{dc223bd47bbe72,
  title = {ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience? — arXiv:2610.07792},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07792},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_223bd47bbe72; sha256 0add03c74376b11cc161b949033a808befc2e0c6c53f27a51792c802adc91941},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07792. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model agents are increasingly deployed to perform complex tasks in real-world environments. However, the knowledge required for correct behavior in these environments is often implicit, undisclosed, and subject to change over time. Recent continual-learning harnesses seek to address this challenge by enabling agents to improve from serving experience. Yet the effectiveness and limitations of these methods are not yet well characterized. Existing benchmarks provide only partial coverage: some explicitly provide the target knowledge, others assume a static environment, and those that support continual adaptation remain limited in scale and knowledge diversity. To enable systematic evaluation, we formalize an evolving-environment streaming dataset (EESD), in which agents must infer, apply, and revise latent environment knowledge from interaction and outcome feedback as hidden policies evolve, and introduce ServeLearnBench, spanning retail support, banking, and sales-pitch generation with 53 environment windows and 7,718 tasks. We evaluate five learning harnesses (RAG, Mem0, SkillOpt, Continual Harness, and Prime) across six models (GPT-5.6 Terra, Opus 5, Kimi K3, GLM-5.3, DeepSeek V4.1 Flash, and GLM-5.3 Flash), covering 28 model-harness pairs and 252 learning runs. Our evaluation reveals three main findings: a substantial gap remains between task capability and learning from experience; continual adaptation is costly and can degrade already-correct behavior; and insufficient exploration emerges as a key bottleneck to effective adaptation. Overall, ServeLearnBench provides a controlled testbed for diagnosing these limitations and tracking progress toward agents that continually and reliably improve through serving experience.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.07792 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:24.767Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).