Metraipolis › archive › dc_aa39755d7e9b

Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC — arXiv:2609.01061

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.01061
License (as archived)
CC0-1.0
Archived
2026-10-17T06:52:37.570Z
SHA-256 of the archived text
c052ebd7f3f5cf1ea7186089443b05994f2733165c61477fa1f1786b1e792057
Stable URL of this page
https://metraipolis.com/documento/dc_aa39755d7e9b

How to cite

Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC — arXiv:2609.01061 — arXiv (en), https://arxiv.org/abs/2609.01061 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa39755d7e9b (Metraipolis archive dc_aa39755d7e9b, sha256 c052ebd7f3f5cf1ea7186089443b05994f2733165c61477fa1f1786b1e792057)
BibTeX
@misc{dcaa39755d7e9b,
  title = {Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC — arXiv:2609.01061},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.01061},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa39755d7e9b; sha256 c052ebd7f3f5cf1ea7186089443b05994f2733165c61477fa1f1786b1e792057},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01061. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In Model Predictive Control (MPC), cost-function weights shape closed-loop behavior, yet changing conditions often make fixed parametrizations suboptimal and motivate context-dependent online adaptation. Learning such policies is difficult because behavior depends implicitly on numerical MPC solutions, producing nonlinear, potentially nonsmooth, long-horizon dependencies on policy parameters. This creates a bias-variance tradeoff: Reinforcement Learning (RL) optimizes realized closed-loop return from environment samples but is sample-inefficient, whereas Gradient-Based Policy Learning (GB-PL) uses low-variance solver gradients from differentiable MPC to optimize surrogate losses on predicted trajectories but can be biased under model mismatch. We propose Solver-Gradient Guided Reinforcement Learning (SG-RL), a solver-sensitivity augmentation for RL-based online MPC cost-weight adaptation. SG-RL keeps sampled closed-loop return as the objective and uses bounded solver-derived gradients as auxiliary guidance to improve stability and sample efficiency. We instantiate SG-RL in Proximal Policy Optimization (PPO) with four modular algorithms that inject solver-gradient guidance into actor-update scaling, policy loss, advantage estimation, and value-function learning. On two full-scale autonomous racing platforms with intentional model mismatch, SG-RL reaches PPO's best closed-loop return with up to 70.6% fewer samples, outperforms GB-PL baselines by at least 54% in closed-loop return, and generalizes zero-shot to unseen environments.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.LG).
Categorie arXiv: Robotics (cs.RO) ; Machine Learning (cs.LG); Systems and Control (eess.SY).
Pagina del lavoro: https://arxiv.org/abs/2609.01061 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:41:55.915Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).