Metraipolis › archive › dc_997eef4e3858

Aligning One-Step Generative Models with Reward-Weighted Transport Distillation — arXiv:2609.30840

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.30840
License (as archived): CC0-1.0
Archived: 2026-10-08T02:10:58.201Z
SHA-256 of the archived text: aa55380d694cf83fca4cd543538d56219a958f305961f7fb4c7df09124f2a893
Stable URL of this page: https://metraipolis.com/documento/dc_997eef4e3858

How to cite

Aligning One-Step Generative Models with Reward-Weighted Transport Distillation — arXiv:2609.30840 — arXiv (en), https://arxiv.org/abs/2609.30840 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_997eef4e3858 (Metraipolis archive dc_997eef4e3858, sha256 aa55380d694cf83fca4cd543538d56219a958f305961f7fb4c7df09124f2a893)

BibTeX

@misc{dc997eef4e3858,
  title = {Aligning One-Step Generative Models with Reward-Weighted Transport Distillation — arXiv:2609.30840},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30840},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_997eef4e3858; sha256 aa55380d694cf83fca4cd543538d56219a958f305961f7fb4c7df09124f2a893},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30840. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

One-step generators enable high-quality visual generation with a single network evaluation, but their post-training is difficult: general implicit generators provide neither tractable likelihoods nor denoising trajectories, and many rewards are non-differentiable. We introduce Reward-Weighted Transport Distillation (RWTD), a post-training method that requires only generated samples and scalar reward evaluations. Rather than aligning solely to the conventional reward-tilted reference distribution, RWTD constructs an adaptive target that mixes separately tilted current and reference distributions. The current component incorporates improvements discovered during training, while the reference component anchors the target to the pretrained generator. RWTD realizes this target through feature-space optimal transport and fixed-point regression. Theoretical analysis shows that the fixed-point distributions of RWTD interpolate between off-policy reward tilting of the reference and on-policy tilting of the current model, providing a principled approach to balancing reward adaptation with retention of prior knowledge. Empirically, RWTD substantially improves the GenEval score of the one-step SANA Sprint 1.6B backbone from 0.73 to 0.80, while separate preference alignment experiments demonstrate strong cross-reward generalization that yields balanced improvements and preservation of compositional capabilities.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.30840 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:43:06.707Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).