Metraipolis › archive › dc_e7e2cc3e633b

Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards — arXiv:2610.22285

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22285
License (as archived)
CC0-1.0
Archived
2026-10-16T09:31:08.761Z
SHA-256 of the archived text
1cbfd0edab3153dd9cd979ddab4a7405f1758ba10c3b657b25be5647b0c55710
Stable URL of this page
https://metraipolis.com/documento/dc_e7e2cc3e633b

How to cite

Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards — arXiv:2610.22285 — arXiv (en), https://arxiv.org/abs/2610.22285 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e7e2cc3e633b (Metraipolis archive dc_e7e2cc3e633b, sha256 1cbfd0edab3153dd9cd979ddab4a7405f1758ba10c3b657b25be5647b0c55710)
BibTeX
@misc{dce7e2cc3e633b,
  title = {Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards — arXiv:2610.22285},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22285},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e7e2cc3e633b; sha256 1cbfd0edab3153dd9cd979ddab4a7405f1758ba10c3b657b25be5647b0c55710},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22285. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reward learning typically assumes the human's input covers everything that matters for the task. In practice, humans can only attend to so much at once, so their teaching naturally captures some features---or aspects of the task---better than others. Prior work has shown how the robot can identify which features were poorly supervised directly from its teaching data. But the current solution is to ask the human to try again, putting the burden on someone who already struggled to convey those features. We propose shifting this burden to the robot. Our key insight is that the same information that tells the robot what it doesn't know can help it ask better questions. Preference queries offer a low-effort channel for resolving this uncertainty, but only if the robot knows where to direct them and the human knows what to attend to when answering. We act on this insight in two ways: we build an informed prior that targets preference queries at the features that need resolving, and we accompany each query with an explanation that focuses the human's attention on exactly those features. In simulation, the informed prior beats the standard active preference learning baseline by up to $30.8\%$ in the low-query regime, where reducing human effort matters most. In a user study, the informed prior and explanations each significantly improve reward recovery and amplify each other.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.22285 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:18.049Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).