Metraipolis › archive › dc_08b71e101897
Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents — arXiv:2609.36393
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36393
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:25:46.128Z
- SHA-256 of the archived text
53e44ed3d03d34d9320b71261d5bd2f82697bd46010aa42388e72a99b6f9a5b4- Stable URL of this page
https://metraipolis.com/documento/dc_08b71e101897
How to cite
Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents — arXiv:2609.36393 — arXiv (en), https://arxiv.org/abs/2609.36393 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_08b71e101897 (Metraipolis archive dc_08b71e101897, sha256 53e44ed3d03d34d9320b71261d5bd2f82697bd46010aa42388e72a99b6f9a5b4)
BibTeX
@misc{dc08b71e101897,
title = {Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents — arXiv:2609.36393},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36393},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_08b71e101897; sha256 53e44ed3d03d34d9320b71261d5bd2f82697bd46010aa42388e72a99b6f9a5b4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36393. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Traditional reinforcement learning (RL) techniques focus on maximizing expected cumulative reward, where each action assumes to take a constant unit of time. However, this assumption does not hold for agentic RL tasks such as machine learning engineering (MLE) agents, where actions involve data loading, feature engineering, and model training that take variable durations. Efficiency matters in modern agentic RL where actions are costly. To address this limitation, we adapt from continuous-time RL and Semi-Markov Decision Process (SMDP) formulation and propose Reward-rate Policy Gradient (RPG), where we focus on optimizing the reward rate -- the long-term reward per unit of time. RPG estimates the reward rate from off-policy samples, then charges each action for the time it consumes at that rate. We first conduct theoretical analysis in the bandit setting to establish that RPG approximates the optimal reward rate and empirically demonstrate it outperforms baselines while avoiding enumeration over the policy space, a known issue for an existing method. We then further apply RPG on a small language model (Qwen3.5-4B) with self-improvement loops and empirically show it obtains higher rewards within a fixed time budget than vanilla RL on MLE-Bench and NanoGPT, with a 19.2% and 85.7% margin, respectively. Our method provides a practical solution for optimizing performance under wait time considerations in modern agentic RL tasks, where actions interact with external environments and cost time. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.36393 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:53.830Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).