Metraipolis › archive › dc_f67896f88230

Perturb the Thought, Not the Pixels: Latent-Space Rollout Diversification for Reinforcement Learning of Vision-Language Models — arXiv:2608.21595

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.21595
License (as archived)
CC0-1.0
Archived
2026-10-18T08:56:36.593Z
SHA-256 of the archived text
33c219baa3c2c686414e2788c3196f3393bf1c091d32c5556b92f4383a517644
Stable URL of this page
https://metraipolis.com/documento/dc_f67896f88230

How to cite

Perturb the Thought, Not the Pixels: Latent-Space Rollout Diversification for Reinforcement Learning of Vision-Language Models — arXiv:2608.21595 — arXiv (en), https://arxiv.org/abs/2608.21595 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f67896f88230 (Metraipolis archive dc_f67896f88230, sha256 33c219baa3c2c686414e2788c3196f3393bf1c091d32c5556b92f4383a517644)
BibTeX
@misc{dcf67896f88230,
  title = {Perturb the Thought, Not the Pixels: Latent-Space Rollout Diversification for Reinforcement Learning of Vision-Language Models — arXiv:2608.21595},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.21595},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f67896f88230; sha256 33c219baa3c2c686414e2788c3196f3393bf1c091d32c5556b92f4383a517644},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21595. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning with verifiable rewards (RLVR) improves the reasoning ability of vision-language models (VLMs), and diversifying the rollouts within each optimization group amplifies its gains. Existing approaches diversify through decoding temperature or pixel-space image distortion; we ask whether the perturbation belongs in the model's latent space instead. We introduce Noise-Contrastive GRPO (NC-GRPO), which injects scale-calibrated Gaussian noise into the last hidden layer of the prompt-encoding pass for half of each rollout group, branching those rollouts from a displaced departure state. Branches that reach the answer despite the displacement are reinforced over those derailed by it, converting sensitivity at the branch point into policy-gradient signal; the objective, reward, and inference protocol are untouched. On Qwen2.5-VL-7B trained on Geometry3K, NC-GRPO significantly improves out-of-domain mathematical reasoning over vanilla GRPO across five held-out benchmarks (pooled McNemar $p \le 0.001$) while also improving in-domain accuracy and hallucination robustness -- the latter an axis on which image-space noise regresses even while posting a larger OOD average on perception-heavy benchmarks. Mechanism ablations indicate that independent stochastic diversity, not noise budget or direction, is the active ingredient, and a noise-scale study exposes a dial between reasoning specialization and general capability. NC-GRPO is designed to be modality-agnostic and integrates into a standard RLVR pipeline as a ~50-line change to the inference engine.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-25 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.21595 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T08:48:09.095Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).