Metraipolis › archive › dc_aa305ba5c8ea

MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors — arXiv:2610.35697

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.35697
License (as archived): CC0-1.0
Archived: 2026-10-27T08:03:24.369Z
SHA-256 of the archived text: c0886e9c6fc3ad8c667083c50edf355ee31f884c1e7da683ac22cd5eb374309d
Stable URL of this page: https://metraipolis.com/documento/dc_aa305ba5c8ea

How to cite

MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors — arXiv:2610.35697 — arXiv (en), https://arxiv.org/abs/2610.35697 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa305ba5c8ea (Metraipolis archive dc_aa305ba5c8ea, sha256 c0886e9c6fc3ad8c667083c50edf355ee31f884c1e7da683ac22cd5eb374309d)

BibTeX

@misc{dcaa305ba5c8ea,
  title = {MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors — arXiv:2610.35697},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.35697},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa305ba5c8ea; sha256 c0886e9c6fc3ad8c667083c50edf355ee31f884c1e7da683ac22cd5eb374309d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35697. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Error-targeted learning gives a legible way to improve a model: identify specific weaknesses of the model, train the model to fix these weaknesses, and check whether the weaknesses are fixed. We study how to apply error-targeted learning to terminal agents, which solve long-horizon tasks specified in natural language by interacting with a command shell. Existing error-targeted methods are designed for single-turn tasks, and existing analyses of agent failures are used to evaluate and debug agents rather than to train them. We propose Misbehavior-driven RL (MiRL), which first profiles the errors of a policy to find generalizable misbehaviors: kinds of wrong steps that recur across many environments. MiRL then trains the policy with RL, starting rollouts at the steps where these misbehaviors occur and rewarding a rollout only if it passes the tests and avoids the misbehavior. On Qwen3.5-4B, eight misbehaviors account for 52.1% of wrong steps. After training with MiRL, the policy repeats the measured misbehaviors 43% to 57% less often on average on unseen prefixes. MiRL outperforms TMax-4B on Terminal-Bench Lite (51.8% vs. 47.4%) and matches it on Terminal-Bench 2.1 (21.3% vs. 20.6%), while using 35% of its rollouts. These results show that error-targeted learning can be applied to realistic, long-horizon terminal tasks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.35697 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:31.011Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).