Metraipolis › archive › dc_108e9e04f830

WAM-OPD: Sharpening World Action Models via On-Policy Distillation — arXiv:2609.34250

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34250
License (as archived)
CC0-1.0
Archived
2026-10-07T21:02:17.883Z
SHA-256 of the archived text
012b1eaa44c19203a075cd0e0a40382fdab168457933145832621b5dda16eb68
Stable URL of this page
https://metraipolis.com/documento/dc_108e9e04f830

How to cite

WAM-OPD: Sharpening World Action Models via On-Policy Distillation — arXiv:2609.34250 — arXiv (en), https://arxiv.org/abs/2609.34250 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_108e9e04f830 (Metraipolis archive dc_108e9e04f830, sha256 012b1eaa44c19203a075cd0e0a40382fdab168457933145832621b5dda16eb68)
BibTeX
@misc{dc108e9e04f830,
  title = {WAM-OPD: Sharpening World Action Models via On-Policy Distillation — arXiv:2609.34250},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34250},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_108e9e04f830; sha256 012b1eaa44c19203a075cd0e0a40382fdab168457933145832621b5dda16eb68},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34250. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Pretrained world action models (WAMs) provide generalist capabilities across diverse robotic manipulation tasks, yet improving target-task performance to an expert level without degrading pretrained skills remains challenging. We explore on-policy distillation (OPD) for WAMs and introduce WAM-OPD. WAM-OPD inherits the advantage of OPD methods that transfer task-specific teacher knowledge under the student's own induced distribution, rather than directly fitting the student to a narrow task-specific data distribution. However, in closed-loop manipulation, the observation histories change as the student policy evolves, requiring fresh environment rollouts to remain on-policy. Applying OPD to WAMs entails repeated data collection, which is costly even in simulation and often impractical on real robots. To avoid repeated environment rollouts during distillation, we introduce prefix-weighted trajectory replay (PWTR). PWTR uses a fixed trajectory pool composed primarily of initial-student rollouts, supplemented with task-specific teacher rollouts to broaden trajectory coverage. For each trajectory replayed from this pool, PWTR conditions the current policy on successive stored histories to generate fresh denoising paths, along which the task-specific teacher provides supervision. Although these denoising paths are refreshed as the policy evolves, the replayed environment trajectories remain fixed. PWTR therefore reweights per-decision distillation losses using proxy importance weights derived from path scores accumulated over the trajectory prefix preceding each decision to mitigate the resulting shift in the history distribution. Simulated and real-world experiments demonstrate task adaptation without additional environment interaction during distillation. In both settings, WAM-OPD improves target-task performance while retaining near-initial performance on tasks excluded from adaptation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.34250 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.908Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).