Metraipolis › archive › dc_cd32226e9903

Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents — arXiv:2608.17524

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17524
License (as archived)
CC0-1.0
Archived
2026-10-18T14:48:39.362Z
SHA-256 of the archived text
71aa8cbf06b6de02cb4e74267b99687662573dce3c36eec8fed5efe09ad391ba
Stable URL of this page
https://metraipolis.com/documento/dc_cd32226e9903

How to cite

Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents — arXiv:2608.17524 — arXiv (en), https://arxiv.org/abs/2608.17524 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cd32226e9903 (Metraipolis archive dc_cd32226e9903, sha256 71aa8cbf06b6de02cb4e74267b99687662573dce3c36eec8fed5efe09ad391ba)
BibTeX
@misc{dccd32226e9903,
  title = {Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents — arXiv:2608.17524},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17524},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cd32226e9903; sha256 71aa8cbf06b6de02cb4e74267b99687662573dce3c36eec8fed5efe09ad391ba},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17524. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This preliminary paper outlines a planned evaluation benchmark for Explainable Reinforcement Learning (XRL) methods. Current evaluations rely on functionally-grounded metrics like faithfulness and compactness, and on human-grounded proxies like subjective ratings or prediction accuracy. We suggest evaluating XRL methods by how effectively their generated explanations help to diagnose and fix malfunctioning reinforcement learning (RL) agents. We propose EvalXRL, a benchmark in which a Large Language Model (LLM) coding agent uses different XRL methods to diagnose a held-out malfunction in an RL agent, and then repair it. Our proposed benchmark iterates across (environment $\times$ malfunction $\times$ XRL method) tuples and uses the reward signal of the RL agents to form a final score for each XRL method. The coding agent may use the method interactively: invoke the XRL method, process its output, form new hypotheses on what is broken, and invoke the method again with parameters adjusted for testing these hypotheses. This closed-loop structure may be described as a simplified version of the scientific method. Some XRL methods provide self-evaluations that follow this pattern; we propose the first head-to-head comparison of multiple XRL methods in closed-loop usage.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.17524 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:47:59.400Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).