Metraipolis › archive › dc_baee5f792b43

Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning — arXiv:2610.20463

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.20463
License (as archived)
CC0-1.0
Archived
2026-10-15T14:20:34.175Z
SHA-256 of the archived text
c18e7507cf6a342d02c295ef6d32d5821809d3dbc548232722664cd5bd17e442
Stable URL of this page
https://metraipolis.com/documento/dc_baee5f792b43

How to cite

Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning — arXiv:2610.20463 — arXiv (en), https://arxiv.org/abs/2610.20463 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_baee5f792b43 (Metraipolis archive dc_baee5f792b43, sha256 c18e7507cf6a342d02c295ef6d32d5821809d3dbc548232722664cd5bd17e442)
BibTeX
@misc{dcbaee5f792b43,
  title = {Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning — arXiv:2610.20463},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.20463},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_baee5f792b43; sha256 c18e7507cf6a342d02c295ef6d32d5821809d3dbc548232722664cd5bd17e442},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20463. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Value-based Q-learning for continuous control can discretise the action space, but the resolution of the discretisation trades early exploration against final precision. Growing Q-Networks (GQN) grow the resolution uniformly from coarse to fine. We introduce Selective Growing Q-Networks (SGQN), which keeps decoupled Q-learning and replaces uniform growth with an adaptive action grid: bins are added next to bins with high estimated Q-value, bins with low Q-value are pruned, and actions are chosen epsilon-greedily. On eight DeepMind Control Suite (DMControl) tasks with 2 to 12 action dimensions, SGQN has the highest mean final return among the compared methods on four, is within 1.1% of the best method on three more, and is 3.4% below Bang-Bang MPO on Walker Run, using on average 7.5 of 9 available bins per dimension. We then test whether the adaptive grid explains these results, and find that it does not. Growth activates every candidate bin and behaves like uniform growth, and removing pruning changes the return by at most 1.1%. Two other choices matter: starting from a coarse grid, since starting with all 9 bins produces collapsed seeds, and epsilon-greedy selection, since an annealed softmax lowers the return by 23% to 100%. SGQN is insensitive to its growth and pruning hyperparameters. It also differs from the GQN and DecQN baselines in other respects, notably one gradient update per environment step instead of one per eight, which we did not control for, so we do not attribute its advantage to a single component. We present SGQN as a competitive value-based method and as an analysis of which ingredients of a growing action grid matter.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.20463 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.758Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).