Metraipolis › archive › dc_29bcb85b104c
Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients — arXiv:2609.22833
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22833
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T07:06:51.239Z
- SHA-256 of the archived text
bd1513af5297497610a0db3ce0f1834038c978e140f2b68e31d9d9e940466604- Stable URL of this page
https://metraipolis.com/documento/dc_29bcb85b104c
How to cite
Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients — arXiv:2609.22833 — arXiv (en), https://arxiv.org/abs/2609.22833 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_29bcb85b104c (Metraipolis archive dc_29bcb85b104c, sha256 bd1513af5297497610a0db3ce0f1834038c978e140f2b68e31d9d9e940466604)
BibTeX
@misc{dc29bcb85b104c,
title = {Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients — arXiv:2609.22833},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22833},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_29bcb85b104c; sha256 bd1513af5297497610a0db3ce0f1834038c978e140f2b68e31d9d9e940466604},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22833. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We study personalized federated reinforcement learning, in which $n$ agents, each acting in its own Markov decision process, collaborate through a server to learn a shared MAML-style policy initialization that becomes effective for an individual agent once that agent adapts it with a single local policy-gradient step. We propose Per-FedAvg-PG, in which agents take $\tau$ local stochastic meta-policy-gradient steps between communication rounds, and prove that it reaches an $\varepsilon$-approximate first-order stationary point of the personalized objective in $K=\mathcal O(\varepsilon^{-3/2})$ rounds with $\tau=\Theta(\varepsilon^{-1/2})$ local steps. The analysis rests on a structural feature of the reinforcement learning setting: under standard policy-class regularity, the per-agent objectives have uniformly bounded gradients and Hessians with explicit constants, so the bounded-gradient and bounded-heterogeneity conditions imposed by the supervised theory hold automatically and no separate heterogeneity assumption is needed. The exact meta-gradient requires the inner-loop policy Hessian, which our experiments identify as the practical bottleneck. We therefore analyze the Hessian-free variant, bound its bias, and exhibit fixed points at which the meta-gradient is nonzero and of order $\alpha$, showing that the resulting stationarity floor is a property of the method rather than of the bound. Experiments on tabular and neural navigation confirm the predicted behavior and show transfer to unseen agents at an order of magnitude lower sample cost than independent training. Together these results identify the adaptation step size as a tunable personalization knob and the curvature estimate as the quantity that governs whether exact meta-gradients are affordable.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2609.22833 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.228Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).