Metraipolis › archive › dc_e7ea87ac1766
Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis — arXiv:2609.33239
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33239
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:39:57.709Z
- SHA-256 of the archived text
0b1b448bd4a82694faf4ba7c148f7f13156760f1cee41694259da6e9078fd490- Stable URL of this page
https://metraipolis.com/documento/dc_e7ea87ac1766
How to cite
Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis — arXiv:2609.33239 — arXiv (en), https://arxiv.org/abs/2609.33239 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e7ea87ac1766 (Metraipolis archive dc_e7ea87ac1766, sha256 0b1b448bd4a82694faf4ba7c148f7f13156760f1cee41694259da6e9078fd490)
BibTeX
@misc{dce7ea87ac1766,
title = {Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis — arXiv:2609.33239},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33239},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e7ea87ac1766; sha256 0b1b448bd4a82694faf4ba7c148f7f13156760f1cee41694259da6e9078fd490},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33239. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion policies represent multimodal action distributions, but an advantage-weighted update does not specify how to sample from the resulting target distribution. We propose Schrödinger--Föllmer Actor--Critic (SFAC), an offline-to-online reinforcement learning (RL) method for Kullback--Leibler (KL)-regularized policy improvement through conditional diffusion. A minimax Bellman critic estimates the advantage function, which defines an exponentially tilted target policy. A Doob $h$-transform expresses this update as a correction to the reference diffusion drift. We derive a posterior-mean representation of the correction and estimate it using paired self-normalized importance sampling (SNIS). Supervised regression on these drift targets updates the neural actor without critic action gradients. In the small-update regime, the KL-regularized update follows the natural policy-gradient direction, and the Doob correction represents the same local change in the space of diffusion drifts. Under suitable conditions, we derive finite-sample bounds that separate the effects of critic estimation, neural drift regression, finite-sample SNIS, diffusion discretization, and inherited actor error on expected average policy suboptimality. Synthetic experiments assess the accuracy of approximation to prescribed advantage-tilted targets and sensitivity to sampling budgets. On six offline-to-online continuous-control tasks, a reference-anchored implementation achieves higher final-window returns than those of its corresponding offline initialization. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.33239 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:28.699Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).