Metraipolis › archive › dc_e992e7055adc

PAC: Progress-Augmented Advantage Curriculum for Multi-Task Reinforcement Learning of LLMs — arXiv:2608.30528

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.30528
License (as archived)
CC0-1.0
Archived
2026-10-17T08:39:16.540Z
SHA-256 of the archived text
09965ce69e67a95e24a517ba9df50e3b4b7cc56caf946e344077e7d4b8cc090c
Stable URL of this page
https://metraipolis.com/documento/dc_e992e7055adc

How to cite

PAC: Progress-Augmented Advantage Curriculum for Multi-Task Reinforcement Learning of LLMs — arXiv:2608.30528 — arXiv (en), https://arxiv.org/abs/2608.30528 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e992e7055adc (Metraipolis archive dc_e992e7055adc, sha256 09965ce69e67a95e24a517ba9df50e3b4b7cc56caf946e344077e7d4b8cc090c)
BibTeX
@misc{dce992e7055adc,
  title = {PAC: Progress-Augmented Advantage Curriculum for Multi-Task Reinforcement Learning of LLMs — arXiv:2608.30528},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.30528},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e992e7055adc; sha256 09965ce69e67a95e24a517ba9df50e3b4b7cc56caf946e344077e7d4b8cc090c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30528. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning (RL) is used to improve the reasoning abilities of LLMs, while training data span heterogeneous tasks. However, most RL post-training pipelines rely on fixed or manually designed task mixtures, even though task usefulness changes as training progresses. Online curriculum methods often define learnability by update magnitude, ignoring whether the update translates into reward gains, which can misallocate rollout budget toward tasks with large but ineffective updates. We propose PAC, a Progress-Augmented Advantage Curriculum for multi-task RL of LLMs that combines two task-level signals: advantage-derived learnability, which measures the magnitude of the policy update a task can induce, and recent reward gains, which show whether those updates have improved task performance. A Bayesian Thompson Sampling controller uses these signals to allocate rollouts across tasks during GRPO training. We evaluate PAC under two settings: a multi-level reasoning setting and a multi-domain reasoning setting. PAC improves sample efficiency and final performance: it reaches comparable validation scores with fewer rollout steps and achieves higher final averages than random sampling and advantage-based curriculum baselines in both settings. These results show that jointly tracking advantage signals and actual reward gains yields an effective online curriculum for LLM post-training.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.30528 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T08:13:26.827Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).