Metraipolis › archive › dc_94ab54d2bbb5

DIA: Denoising Intermediate Advantage for Diffusion Policy Optimization — arXiv:2609.12245

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12245
License (as archived)
CC0-1.0
Archived
2026-10-16T03:58:06.498Z
SHA-256 of the archived text
bb7e0de809e1e2021736a686e38f2b6b2344bbab23f52c4a039fcfb089d53996
Stable URL of this page
https://metraipolis.com/documento/dc_94ab54d2bbb5

How to cite

DIA: Denoising Intermediate Advantage for Diffusion Policy Optimization — arXiv:2609.12245 — arXiv (en), https://arxiv.org/abs/2609.12245 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_94ab54d2bbb5 (Metraipolis archive dc_94ab54d2bbb5, sha256 bb7e0de809e1e2021736a686e38f2b6b2344bbab23f52c4a039fcfb089d53996)
BibTeX
@misc{dc94ab54d2bbb5,
  title = {DIA: Denoising Intermediate Advantage for Diffusion Policy Optimization — arXiv:2609.12245},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12245},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_94ab54d2bbb5; sha256 bb7e0de809e1e2021736a686e38f2b6b2344bbab23f52c4a039fcfb089d53996},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12245. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion-based robot policies have become widely used in robotic manipulation, where they are typically trained with behavior cloning. However, policies trained purely from demonstrations are limited by the quality and coverage of the available data. Reinforcement learning can further improve the performance of these pretrained policies through interaction. A common approach is to use policy-gradient methods that formulate diffusion-policy fine-tuning as an outer environment MDP together with an inner denoising MDP. However, existing methods typically assign the same environment-level credit to all denoising steps used to construct an action chunk, without distinguishing which intermediate decisions contributed most to the final return. We introduce Denoising Intermediate Advantage (DIA), a policy-gradient method that learns a value function over partially denoised actions and uses it to construct a denoising level advantage for each step of the generative process. DIA combines this inner credit signal with the standard environment-level PPO advantage, providing state-dependent credit throughout the denoising chain. Across Robomimic, FurnitureBench, Franka Kitchen, and D3IL, DIA consistently improves final performance over existing diffusion-policy fine-tuning methods. Beyond final reward, DIA reaches successful states more efficiently and can shift farther from the pretrained behavior distribution, enabling it to discover more effective and efficient task-level strategies and subtask sequences that baseline methods fail to reach.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-14 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.12245 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:44:44.704Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).