Metraipolis › archive › dc_d2eb81019398

Recovering Aggressively Pruned Vision-Language-Action Models with Offline Hidden-State Distillation — arXiv:2609.19579

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.19579
License (as archived)
CC0-1.0
Archived
2026-10-10T05:25:27.543Z
SHA-256 of the archived text
eb115265d367ce426d5e02dedbcc32050b59647aeb40266e3bad46080492cafd
Stable URL of this page
https://metraipolis.com/documento/dc_d2eb81019398

How to cite

Recovering Aggressively Pruned Vision-Language-Action Models with Offline Hidden-State Distillation — arXiv:2609.19579 — arXiv (en), https://arxiv.org/abs/2609.19579 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d2eb81019398 (Metraipolis archive dc_d2eb81019398, sha256 eb115265d367ce426d5e02dedbcc32050b59647aeb40266e3bad46080492cafd)
BibTeX
@misc{dcd2eb81019398,
  title = {Recovering Aggressively Pruned Vision-Language-Action Models with Offline Hidden-State Distillation — arXiv:2609.19579},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19579},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d2eb81019398; sha256 eb115265d367ce426d5e02dedbcc32050b59647aeb40266e3bad46080492cafd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19579. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language-action (VLA) models let robots follow language instructions, but their language backbones of several billion parameters are the main obstacle to running them on robot hardware. Structured pruning reduces that backbone, and removing 63% of it from OpenVLA-OFT drops LIBERO-Long success from 93.2% to 0.8%. A recent approach restores such a model with supervised fine-tuning followed by reinforcement learning, which needs online rollouts and hundreds of GPU-hours. We recover most of the lost success entirely offline. Width pruning narrows the blocks but keeps the residual stream at its original size, so teacher and student hidden states have the same shape and are matched directly, without a projector. Training against a cache built in one teacher pass lifts the 63%-reduced student to within 3.5 points of the teacher in about 8 GPU-hours. A sweep over nine ratios locates where the recovery objective starts to matter. Up to 45% reduction the two do not differ significantly on OpenVLA-OFT. Hidden-state distillation then adds +2.1 to +4.5 points there between 63% and 87%, and +9.4 to +22.1 points on CogACT from 63% onward. At 81% on CogACT, a tripled recovery budget narrows the distilled student's gap to the teacher to 3.9 points on average, while supervised recovery stays more than 20 points below. At matched compression, width pruning yields higher success and depth pruning lower latency. On a 6-DoF manipulator, the distilled student at 72% reduction reaches 77.5% success against 59.5% for supervised recovery, runs 2.23x faster on-board than the teacher, and uses 62% less memory.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-18 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.19579 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T04:47:38.833Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).