Metraipolis › archive › dc_be63c890e848
Beyond Oracle Communication: Benchmarking Interactive Intent Alignment Under Miscommunication and Evolving User Intent — arXiv:2609.38604
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38604
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T12:43:58.681Z
- SHA-256 of the archived text
5bbee0c102aa651a81a4ea4743347f9d78dd8d56acf398acbdaa28109575e5e5- Stable URL of this page
https://metraipolis.com/documento/dc_be63c890e848
How to cite
Beyond Oracle Communication: Benchmarking Interactive Intent Alignment Under Miscommunication and Evolving User Intent — arXiv:2609.38604 — arXiv (en), https://arxiv.org/abs/2609.38604 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_be63c890e848 (Metraipolis archive dc_be63c890e848, sha256 5bbee0c102aa651a81a4ea4743347f9d78dd8d56acf398acbdaa28109575e5e5)
BibTeX
@misc{dcbe63c890e848,
title = {Beyond Oracle Communication: Benchmarking Interactive Intent Alignment Under Miscommunication and Evolving User Intent — arXiv:2609.38604},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38604},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_be63c890e848; sha256 5bbee0c102aa651a81a4ea4743347f9d78dd8d56acf398acbdaa28109575e5e5},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38604. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern LLM agents increasingly tackle complex tasks through interactive, long-horizon exchanges with users, while existing benchmarks generally assume that users always accurately and sufficiently communicate a fixed intent. However, this oracle communication assumption rarely holds in practice: users may miscommunicate, change their goals, and run out of patience. We define this task setting as Interactive Intent Alignment, where agents must recover and continuously track the user's current intent despite imperfect communication and evolving goals. To study this setting, we introduce Drift-Bench++, a principled benchmark construction pipeline for verified executable tasks with controlled misalignment and intent shifts, along with an interaction protocol featuring finite patience, diverse simulated users, and silent interaction-conditioned shifts. We further develop GRIP, a comprehensive evaluation protocol covering task grounding, user realism, inquiry effectiveness, and adaptation to evolving intent. Across diverse environments, models, and interaction conditions, stronger interaction consistently helps but remains far from oracle performance; Validation on deployed ProdAgent sessions further shows that the modeled failures are prevalent and consequential in deployment. By providing a unified, executable benchmark for interactive intent alignment, Drift-Bench++ offers a foundation for evaluating and advancing agents under realistic communication and evolving intent. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.38604 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:14:13.906Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).