Metraipolis › archive › dc_ef5fd08bcffe

Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces — arXiv:2609.18587

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.18587
License (as archived): CC0-1.0
Archived: 2026-10-15T15:04:48.084Z
SHA-256 of the archived text: 4dd32d449413ecd6107332fe7b46828b3c99a450239adc0eb86ca1c0ab7d2fa6
Stable URL of this page: https://metraipolis.com/documento/dc_ef5fd08bcffe

How to cite

Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces — arXiv:2609.18587 — arXiv (en), https://arxiv.org/abs/2609.18587 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef5fd08bcffe (Metraipolis archive dc_ef5fd08bcffe, sha256 4dd32d449413ecd6107332fe7b46828b3c99a450239adc0eb86ca1c0ab7d2fa6)

BibTeX

@misc{dcef5fd08bcffe,
  title = {Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces — arXiv:2609.18587},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.18587},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef5fd08bcffe; sha256 4dd32d449413ecd6107332fe7b46828b3c99a450239adc0eb86ca1c0ab7d2fa6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18587. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Test-time reinforcement learning (TTRL) enables models to improve their reasoning without relying on labeled training data, but existing approaches typically optimize a large fraction of the model parameters. This raises a natural question: can effective test-time adaptation emerge when both the reward signal and the optimization space are severely restricted? We answer this question with label-free bias-only TTRL, which uses majority-vote pseudolabels as rewards and optimizes only ~100K bias parameters while keeping the pretrained backbone frozen. On MATH-500, our approach reaches 76.67% accuracy with Qwen2.5-7B, slightly exceeding our own labeled bias-steering reproduction while optimizing 76,000x fewer parameters than full-parameter TTRL. The same training procedure improves performance across vision-language and audio reasoning tasks, including MathVista, AI2D, LogicVista, and MMAU. We further show that the learned steering vectors transfer to 4,500 held-out MATH problems, indicating that the adaptation is not limited to the problems used during test-time optimization. Finally, we analyze why this highly restricted adaptation can work, showing that majority-vote reliability improves with rollout consensus and that bias subspaces with greater accessible gradient energy exhibit stronger downstream trainability. These results demonstrate that substantial test-time adaptation can emerge from optimizing a tiny bias-only subspace using entirely label-free rewards.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.18587 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.134Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).