Metraipolis › archive › dc_7ef227221283

MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development — arXiv:2609.36679

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36679
License (as archived)
CC0-1.0
Archived
2026-10-07T16:22:53.316Z
SHA-256 of the archived text
43ebf6ad406eddb44c5f37f81870c6a9a5f1e9292470358e952c053b966c3b10
Stable URL of this page
https://metraipolis.com/documento/dc_7ef227221283

How to cite

MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development — arXiv:2609.36679 — arXiv (en), https://arxiv.org/abs/2609.36679 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ef227221283 (Metraipolis archive dc_7ef227221283, sha256 43ebf6ad406eddb44c5f37f81870c6a9a5f1e9292470358e952c053b966c3b10)
BibTeX
@misc{dc7ef227221283,
  title = {MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development — arXiv:2609.36679},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36679},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ef227221283; sha256 43ebf6ad406eddb44c5f37f81870c6a9a5f1e9292470358e952c053b966c3b10},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36679. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Machine learning engineering (MLE) agents have made substantial progress, but learning through ML experimentation remains costly in time and computation. Synthetic environments reduce these costs while introducing variations in data and experimental settings that require task-specific diagnosis. Access to diagnostic tools alone does not ensure that agents learn when to use them or how to act on their findings. We introduce ToolMLBench, a suite of executable tools for data inspection, code verification, and experiment diagnosis, together with an SFT and RL pipeline for learning their use. Diagnostic calls acquire evidence whose value depends on subsequent decisions, so final outcomes provide limited guidance on which calls to reinforce. We address this challenge with SPICE, which measures how privileged context changes the likelihood of a sampled tool action and uses this difference as a turn-level reward alongside the final outcome. We train on 80 synthetic tasks and evaluate on 25 in-domain and 10 out-of-domain tasks. Providing tool interfaces and descriptions alone yields inconsistent gains across unadapted models. With the same diagnostic interface, our training pipeline raises in-domain success from 24.8% to 52.4% for Qwen3-8B and from 35.6% to 69.2% for Qwen3.5-35B-A3B. The latter also improves from 31% to 48% out-of-domain, supporting learned diagnostic tool use on held-out sources and targets.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.36679 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.527Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).