Metraipolis › archive › dc_f44cdb059ff3

Iterative Gumbel Planning for Continuous Control — arXiv:2611.02236

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02236
License (as archived): CC0-1.0
Archived: 2026-11-03T10:57:46.715Z
SHA-256 of the archived text: 56dfb8b07965a7c59c9758250381d02e48f1bc3e9123d9a222459daf7487c396
Stable URL of this page: https://metraipolis.com/documento/dc_f44cdb059ff3

How to cite

Iterative Gumbel Planning for Continuous Control — arXiv:2611.02236 — arXiv (en), https://arxiv.org/abs/2611.02236 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f44cdb059ff3 (Metraipolis archive dc_f44cdb059ff3, sha256 56dfb8b07965a7c59c9758250381d02e48f1bc3e9123d9a222459daf7487c396)

BibTeX

@misc{dcf44cdb059ff3,
  title = {Iterative Gumbel Planning for Continuous Control — arXiv:2611.02236},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02236},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f44cdb059ff3; sha256 56dfb8b07965a7c59c9758250381d02e48f1bc3e9123d9a222459daf7487c396},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02236. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Search-based planning remains a powerful policy improvement paradigm in reinforcement learning, driving milestone successes in discrete domains. However, extending this paradigm to continuous control exposes a structural mismatch: search evaluates a finite set of action candidates, whereas continuous RL typically relies on Gaussian policies. We show that winner-action distillation into a continuous Gaussian induces a \textit{projection error} and a stepwise variance-expanding gradient tendency when the selected action is sufficiently far from the current mean. Empirically, this tendency is associated with elevated terminal variance and a return gap on the evaluated tasks. To address this, we propose \textbf{Iterative Gumbel Planning (IGP)}. By adopting a per-dimension factorized categorical policy, IGP naturally aligns the policy support with the search candidates, eliminating the projection error and reducing the residual approximation to a bounded discretization error. To bypass the combinatorial explosion of high-dimensional discrete action spaces, IGP pairs Gumbel Top-$K$ candidate sampling with an MPPI-style multi-round iterative search to refine the policy at each state. Empirically, IGP achieves state-of-the-art aggregate performance and sample efficiency on DMControl and HumanoidBench. Furthermore, it produces more stable action sequences with lower temporal variance, demonstrating its practical potential for real-world robotic deployment.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02236 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.632Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).