Metraipolis › archive › dc_f51170ff5e42

What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor — arXiv:2609.32532

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32532
License (as archived)
CC0-1.0
Archived
2026-10-07T21:00:21.115Z
SHA-256 of the archived text
3fe85531e633f5f4d8c7a71c1577321d7782ce190895c03f64b5399caba20775
Stable URL of this page
https://metraipolis.com/documento/dc_f51170ff5e42

How to cite

What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor — arXiv:2609.32532 — arXiv (en), https://arxiv.org/abs/2609.32532 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f51170ff5e42 (Metraipolis archive dc_f51170ff5e42, sha256 3fe85531e633f5f4d8c7a71c1577321d7782ce190895c03f64b5399caba20775)
BibTeX
@misc{dcf51170ff5e42,
  title = {What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor — arXiv:2609.32532},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32532},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f51170ff5e42; sha256 3fe85531e633f5f4d8c7a71c1577321d7782ce190895c03f64b5399caba20775},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32532. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A generalization gap in reinforcement learning, return on training levels minus return on held-out levels, is usually reported without a reference point. We argue that it should be read against a measured random floor: the return of a uniform-random policy on the same levels under the same evaluation harness. On eight ProcGen environments with PPO at a compute-limited budget (8M steps, 16 parallel environments; three games extended to 25M), the floor changes what standard numbers mean. The test-time action rule decides which policy is measured: in miner, the sampled policy scores 5.1x the floor on held-out levels while its argmax scores below it in every run, and greedy evaluation places two environments significantly below the floor. Used as a convergence diagnostic, raw policy entropy flags six of eight environments, but 32-66% of that entropy lies on actions with identical effects; against the floor, five of eight sampled policies are clearly above it on held-out levels and heist's is not distinguishable from it. An audit of twelve ProcGen codebases finds that nine sample test-time actions with no explicit choice at the evaluation call site. We recommend that every reported gap state its action rule, seed its evaluation and specify its tests before analysis, and report the floor on both level sets.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.32532 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.372Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).