Metraipolis › archive › dc_694eeae0ff07

Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation — arXiv:2609.09135

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.09135
License (as archived): CC0-1.0
Archived: 2026-10-16T12:58:37.334Z
SHA-256 of the archived text: d039f0dbe2577bbe8fee1237e314b9af0f82cb73471e0f05c567802b64564d88
Stable URL of this page: https://metraipolis.com/documento/dc_694eeae0ff07

How to cite

Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation — arXiv:2609.09135 — arXiv (en), https://arxiv.org/abs/2609.09135 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_694eeae0ff07 (Metraipolis archive dc_694eeae0ff07, sha256 d039f0dbe2577bbe8fee1237e314b9af0f82cb73471e0f05c567802b64564d88)

BibTeX

@misc{dc694eeae0ff07,
  title = {Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation — arXiv:2609.09135},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09135},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_694eeae0ff07; sha256 d039f0dbe2577bbe8fee1237e314b9af0f82cb73471e0f05c567802b64564d88},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09135. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing methods for test-time reinforcement learning (TTRL) derive rewards from answer-level self-voting on unlabeled test-time tasks with canonical answers, but this breaks down for code generation because programs cannot be compared by surface form and therefore do not directly provide a usable training signal. To make TTRL applicable to code generation, we propose probe-driven TTRL, which constructs output-free probe inputs from the problem statement, executes candidate programs on these probes, and defines a Probe Consensus Reward (PCR) from the resulting behavioral agreement. PCR provides a behavioral training signal for open-vocabulary programs, but it is not a fully reliable verifier and remains susceptible to reward hacking through spurious consensus. We therefore introduce Entropy-Regularized Rank-Masked Policy Optimization (ERPO), which converts low PCR into conservative negative updates through rank masking and controls policy drift with an entropy ceiling. On coding benchmarks, ERPO substantially improves pass@1 and pass@k in both in-domain adaptation and zero-shot transfer.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.09135 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.386Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).