Metraipolis › archive › dc_7ea916b1998b

When Do Options Help? Policy Necrosis and Redundant Coverage in Option-Critic — arXiv:2609.05508

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.05508
License (as archived): CC0-1.0
Archived: 2026-10-16T13:56:14.793Z
SHA-256 of the archived text: 3bab5e18024b28fc3c4147f187cff2efb3376ce7050789d60b94f66f432c115e
Stable URL of this page: https://metraipolis.com/documento/dc_7ea916b1998b

How to cite

When Do Options Help? Policy Necrosis and Redundant Coverage in Option-Critic — arXiv:2609.05508 — arXiv (en), https://arxiv.org/abs/2609.05508 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ea916b1998b (Metraipolis archive dc_7ea916b1998b, sha256 3bab5e18024b28fc3c4147f187cff2efb3376ce7050789d60b94f66f432c115e)

BibTeX

@misc{dc7ea916b1998b,
  title = {When Do Options Help? Policy Necrosis and Redundant Coverage in Option-Critic — arXiv:2609.05508},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.05508},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ea916b1998b; sha256 3bab5e18024b28fc3c4147f187cff2efb3376ce7050789d60b94f66f432c115e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05508. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Option-critic learns options: sub-policies together with a learned rule for when each one hands control back. Its headline result is that performance improves as options are added. We explain that result, with theory and experiment. First, the termination rule option-critic learns by maximising return contributes nothing. When the termination test and the policy that picks options read the same values, the test fires at every step, so the learned rule is identical to always terminating. When that policy explores and the test does not, as in option-critic itself, the rule can block the exploration; there are instances where it suffers $\Omega(T)$ regret while always terminating holds to $O(\log T)$. Forcing termination at every step leaves the option-count curve intact. Second, the policy inside an option barely explores at all, so a state locks onto the first action that looked good and never updates again. We name this policy necrosis, give a state-level test for it, and find three fifths of states necrotic in a typical option. Restoring exploration repairs those states, and one option then solves the task. Third, extra options improve no option; what falls is the chance that all of them fail in the same state, from $59\\%$ to $4\\%$, and performance follows that joint quantity.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.05508 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:39:42.792Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).