Metraipolis › archive › dc_753dddbdb072

What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization — arXiv:2610.19731

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.19731
License (as archived): CC0-1.0
Archived: 2026-10-15T14:17:44.338Z
SHA-256 of the archived text: 47c9f324c1578fc7105122ba46071c9cee22075c87ac2d7010ac727d2f3ee818
Stable URL of this page: https://metraipolis.com/documento/dc_753dddbdb072

How to cite

What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization — arXiv:2610.19731 — arXiv (en), https://arxiv.org/abs/2610.19731 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_753dddbdb072 (Metraipolis archive dc_753dddbdb072, sha256 47c9f324c1578fc7105122ba46071c9cee22075c87ac2d7010ac727d2f3ee818)

BibTeX

@misc{dc753dddbdb072,
  title = {What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization — arXiv:2610.19731},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.19731},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_753dddbdb072; sha256 47c9f324c1578fc7105122ba46071c9cee22075c87ac2d7010ac727d2f3ee818},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19731. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Goal misgeneralization is a common failure mode in which an agent behaves competently out of distribution but does not pursue the intended goal. However, we still have a weak understanding of what happens to the intended goal in misgeneralizing agents. Across three Procgen environments and five robot tasks, we find three distinct forms of policy-goal misalignment: (1) functional target substitution, where behavior follows an alternative goal, (2) a policy shortcut, where behavior follows a learned action strategy, and (3) cue competition between intended and proxy goal cues. We distinguish these regimes by tracing where goal information remains available internally, how strongly it influences the policy, and what role it plays in action selection, and find that the goal is lost at different depths. Where the goal reaches the decision, as in cue competition, steering redirects the policy to the goal's location, raising intended-goal reach from 16.0% to 95.3%. Where it reaches the policy but not the decision, only new downstream wiring recovers goal reach, from 31.6% to 84.3% in the environment with the action shortcut. Where it is inert, neither works. Our results show that goal misgeneralization is not a single failure of goal representation, but distinct failures of policy alignment at different depths, tightly connected to which recovery works.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.19731 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.635Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).