Metraipolis › archive › dc_f02de8a1c2cb

Don't Guess, Ask Wisely: Benchmarking and Training Proactive Elicitation in Open-Domain LLM Conversations — arXiv:2611.01878

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01878
License (as archived)
CC0-1.0
Archived
2026-11-03T18:23:25.817Z
SHA-256 of the archived text
897c642a731522ee815be93e8dc93c3fd305878937331fd82c3f9a058d8b477d
Stable URL of this page
https://metraipolis.com/documento/dc_f02de8a1c2cb

How to cite

Don't Guess, Ask Wisely: Benchmarking and Training Proactive Elicitation in Open-Domain LLM Conversations — arXiv:2611.01878 — arXiv (en), https://arxiv.org/abs/2611.01878 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f02de8a1c2cb (Metraipolis archive dc_f02de8a1c2cb, sha256 897c642a731522ee815be93e8dc93c3fd305878937331fd82c3f9a058d8b477d)
BibTeX
@misc{dcf02de8a1c2cb,
  title = {Don't Guess, Ask Wisely: Benchmarking and Training Proactive Elicitation in Open-Domain LLM Conversations — arXiv:2611.01878},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01878},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f02de8a1c2cb; sha256 897c642a731522ee815be93e8dc93c3fd305878937331fd82c3f9a058d8b477d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01878. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When user queries are incomplete, large language models should ask targeted questions before committing to an answer, a capability we call proactive elicitation, yet evaluation on real user-assistant queries across seven LLMs (8B to frontier) shows that fewer than 11% of underspecified queries trigger a follow-up question. To benchmark and train this capability, we introduce VagueBench, where each seed query is annotated with structured intent and detail units then degraded into intent-vague and fully-vague variants via controlled removal of linguistic cues, yielding known hidden information sets for evaluating both when and what to ask. We propose DialogGRPO, an offline reinforcement learning method that organizes multi-turn elicitation as a dialogue-state directed acyclic graph, where candidate questions at each node receive dense turn-level rewards combining immediate information gain with lookahead value through group-relative comparison, enabling fine-grained credit assignment without runtime simulation. DialogGRPO elicits missing information on over 88% of vague queries with substantial gains in both recovery efficiency and answer coverage, without over-asking on complete queries, outperforms prompting and test-time-scaling baselines, and generalizes zero-shot to MultiWOZ 2.4 and AmbigQA.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.01878 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:07.942Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).