Metraipolis › archive › dc_b6502f8a011b

StructRL: Structured Action-Space Exploration for Flow-Based VLAs — arXiv:2608.15139

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.15139
License (as archived)
CC0-1.0
Archived
2026-10-18T23:05:47.643Z
SHA-256 of the archived text
2c137f3ccfb499c85ba654b60f367895fa968745741ec3dcfc32cbc2856a4743
Stable URL of this page
https://metraipolis.com/documento/dc_b6502f8a011b

How to cite

StructRL: Structured Action-Space Exploration for Flow-Based VLAs — arXiv:2608.15139 — arXiv (en), https://arxiv.org/abs/2608.15139 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b6502f8a011b (Metraipolis archive dc_b6502f8a011b, sha256 2c137f3ccfb499c85ba654b60f367895fa968745741ec3dcfc32cbc2856a4743)
BibTeX
@misc{dcb6502f8a011b,
  title = {StructRL: Structured Action-Space Exploration for Flow-Based VLAs — arXiv:2608.15139},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15139},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b6502f8a011b; sha256 2c137f3ccfb499c85ba654b60f367895fa968745741ec3dcfc32cbc2856a4743},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15139. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Flow-based Vision-Language-Action (VLA) models are now widely used for continuous robotic manipulation, and online reinforcement learning (RL) is emerging as a key technique for adapting them to new tasks. Existing RL methods typically inject stochasticity inside the denoising chain, often through isotropic or temporally independent noise. However, effective robot exploration calls for structured noise: temporally smooth and scaled differently across action groups. We show that simply switching the in-chain noise to a structured form does not suffice: noise added at an intermediate flow time can be weakened by the remaining denoising steps before execution, a phenomenon we call \emph{Structured Noise Dilution}. We propose \textbf{StructRL}, which avoids dilution by relocating policy stochasticity to the action space via three coupled choices: (i) a deterministic ODE decoder, (ii) structured noise injected directly in the action space, and (iii) last-step replay, where policy-gradient updates avoid assigning likelihoods to intermediate denoising states. This keeps structured exploration tied to the executed action while providing a tractable training signal for the flow decoder. Across three flow-based VLA models on multiple simulated manipulation benchmarks and two real-world tasks, StructRL improves exploration efficiency and OOD performance over prior in-chain baselines, demonstrating the effectiveness of structured action-space exploration for adapting flow-based VLA with RL. \textbf{Project page:} this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-18 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2608.15139 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T23:01:50.345Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).