Metraipolis › archive › dc_def8a8dbfb61

LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL — arXiv:2609.03528

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.03528
License (as archived): CC0-1.0
Archived: 2026-10-17T01:11:55.870Z
SHA-256 of the archived text: 39a12de1514fa45b0dc4094070020ed3a082463058ef6a205c2787bdf174ccf2
Stable URL of this page: https://metraipolis.com/documento/dc_def8a8dbfb61

How to cite

LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL — arXiv:2609.03528 — arXiv (en), https://arxiv.org/abs/2609.03528 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_def8a8dbfb61 (Metraipolis archive dc_def8a8dbfb61, sha256 39a12de1514fa45b0dc4094070020ed3a082463058ef6a205c2787bdf174ccf2)

BibTeX

@misc{dcdef8a8dbfb61,
  title = {LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL — arXiv:2609.03528},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.03528},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_def8a8dbfb61; sha256 39a12de1514fa45b0dc4094070020ed3a082463058ef6a205c2787bdf174ccf2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.03528. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diffusion reinforcement learning (RL) has recently achieved significant success in post-training image and video generative models. However, most diffusion RL methods, including DanceGRPO and FlowGRPO, recompute selected timesteps with gradient tracking after rollout. Under on-policy training with the same backend for rollout and update, this recomputation is mathematically redundant. Intuitively, the rollout and policy update steps can reuse the same feed-forward backbone to avoid redundant computation, but doing so can incur a large memory overhead during rollout. To address the issue, we present LeanGRPO by restructuring the data-parallel layout and introducing two recompute-free training schedules for trajectory-logprob diffusion RL: (1) LeanGRPO-Retain enables gradient tracking during rollout and directly reuses the resulting computation graphs and saved activations for backward during update, requiring no recomputation; and (2) LeanGRPO-Reweight also enables gradients during rollout, but immediately backpropagates each selected step using a provisional advantage and delays gradient synchronization, then corrects the provisional gradients with the true advantage after the trajectory is completed. These schedules target different model scales and input sizes. Across FlowGRPO/DanceGRPO with FLUX.1-dev and Wan, LeanGRPO achieves up to 1.83x end-to-end speedup while preserving the original optimization objective.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2609.03528 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T00:28:02.973Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).