Metraipolis › archive › dc_6b7dfeeb7769

RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers — arXiv:2609.34210

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34210
License (as archived)
CC0-1.0
Archived
2026-10-08T00:17:19.438Z
SHA-256 of the archived text
62ee1d1cc5231c8d8b1c1e83f5fc00afd6f03ccbc7d2273565139744f6e819b2
Stable URL of this page
https://metraipolis.com/documento/dc_6b7dfeeb7769

How to cite

RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers — arXiv:2609.34210 — arXiv (en), https://arxiv.org/abs/2609.34210 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6b7dfeeb7769 (Metraipolis archive dc_6b7dfeeb7769, sha256 62ee1d1cc5231c8d8b1c1e83f5fc00afd6f03ccbc7d2273565139744f6e819b2)
BibTeX
@misc{dc6b7dfeeb7769,
  title = {RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers — arXiv:2609.34210},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34210},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6b7dfeeb7769; sha256 62ee1d1cc5231c8d8b1c1e83f5fc00afd6f03ccbc7d2273565139744f6e819b2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34210. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Coding agents are beginning to move beyond purely digital tasks to tackle physical-world challenges, particularly in robotics. Existing robotics benchmarks, however, primarily focus on the performance of individual artifacts, such as policies or controllers, offering limited coverage of coding agents' broader engineering capabilities. Real-world robotics extends beyond control: agents must build, integrate, diagnose, and improve heterogeneous artifacts under resource constraints and reason from multimodal feedback. To evaluate these broader capabilities, we introduce RLE-Bench, a benchmark of robot-learning tasks spanning four representative robotics development workflows: interactive control, policy learning, perception and estimation, and mechanical design. We use diverse task-specific metrics to evaluate the artifacts submitted by the coding agents, from the success rate the agents achieved to the policy agents trained, the harness agent built, and the mechanical structures the agent designed. We aggregate these metrics into an overall RLE Index and report workflow-specific capability profiles, enabling systematic comparison of coding agents' capabilities across multiple capability dimensions. Beyond performance ranks, we also conduct in-depth case studies examining agent behavior on representative tasks, highlighting both current capabilities and limitations, and pointing to the opportunities robotics tasks have to offer for future agent training.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.34210 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T23:52:16.526Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).