Metraipolis › archive › dc_fc114e602efd

AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces — arXiv:2608.23041

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.23041
License (as archived): CC0-1.0
Archived: 2026-10-18T07:41:47.686Z
SHA-256 of the archived text: c2d1ffa78a4811d6fff20f990b0ce4b35e011ef73edf2fdd6565d77ce0e0b95a
Stable URL of this page: https://metraipolis.com/documento/dc_fc114e602efd

How to cite

AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces — arXiv:2608.23041 — arXiv (en), https://arxiv.org/abs/2608.23041 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc114e602efd (Metraipolis archive dc_fc114e602efd, sha256 c2d1ffa78a4811d6fff20f990b0ce4b35e011ef73edf2fdd6565d77ce0e0b95a)

BibTeX

@misc{dcfc114e602efd,
  title = {AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces — arXiv:2608.23041},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.23041},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc114e602efd; sha256 c2d1ffa78a4811d6fff20f990b0ce4b35e011ef73edf2fdd6565d77ce0e0b95a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.23041. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM agents remain unreliable on long-horizon tasks, where small local failures can compound over extended interactions and lead to overall task failure. Although external harnesses can substantially improve robustness, harness design remains a manual and expensive process that requires searching over a large space of prompts, tool configurations, and control logic. We propose AutoSaddler, an automatic harness optimization framework that formulates harness improvement as an offline learning problem and iteratively updates the harness using failure signals from mini-batches. AutoSaddler combines failure-trace diagnosis, structured patch generation that treats the harness as code, and validation-based update selection. Experiments on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 show that AutoSaddler substantially improves agent performance over the corresponding base harnesses, achieving gains of 9.0, 9.6, and 10.0 percentage points, respectively. Ablation studies further suggest that effective harness optimization benefits from three ingredients: deep debugging rather than shallow reflection, targeted modifications rather than unconstrained editing, and generalization-aware selection rather than trajectory-specific repair. Together, these results suggest that automatic harness optimization is a promising path toward more performant and reliable agent systems.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.23041 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.761Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).