Metraipolis › archive › dc_de36df184c22

Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps — arXiv:2610.39264

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39264
License (as archived)
CC0-1.0
Archived
2026-10-31T06:09:13.934Z
SHA-256 of the archived text
5ee08a6a6bfafc8666897f167b8a0f819310dd71923fc3029dabb7c9bb6ad725
Stable URL of this page
https://metraipolis.com/documento/dc_de36df184c22

How to cite

Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps — arXiv:2610.39264 — arXiv (en), https://arxiv.org/abs/2610.39264 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_de36df184c22 (Metraipolis archive dc_de36df184c22, sha256 5ee08a6a6bfafc8666897f167b8a0f819310dd71923fc3029dabb7c9bb6ad725)
BibTeX
@misc{dcde36df184c22,
  title = {Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps — arXiv:2610.39264},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39264},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_de36df184c22; sha256 5ee08a6a6bfafc8666897f167b8a0f819310dd71923fc3029dabb7c9bb6ad725},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39264. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use. Conversely, smaller models trained via supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories, leading to brittle execution and an inability to recover from errors. To address these limitations, we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications. Our framework decouples the agent into two specialized components: a Planning agent and a UI Grounding model. We rely on an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion. To effectively leverage this dual reward structure, we optimize the agent using Proximal Policy Optimization (PPO) with step-level advantage estimation. Furthermore, we introduce GiGPPO, a novel neighbor-conditioned value estimation method that improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories. To support this online training paradigm at scale, we introduce an efficient infrastructure that orchestrates hundreds of concurrent virtual iOS devices. Evaluations on rigorous benchmarks, including UINavBench, iOSWorld, and AndroidWorld, demonstrate that our online RL trained agents substantially outperform both SFT and terminal-only RL baselines while accelerating training convergence. Notably, our 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments, while outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro on real iOS applications.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.39264 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.511Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).