Metraipolis › archive › dc_221befcf886f

You Know What I Mean: A Benchmark for Agentic Conversational Reference Grounding — arXiv:2608.29834

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29834
License (as archived)
CC0-1.0
Archived
2026-10-17T08:07:10.238Z
SHA-256 of the archived text
dc0daf01a68295e168247df33b6528291632bc55d50795ff05b8b1e4ab91c675
Stable URL of this page
https://metraipolis.com/documento/dc_221befcf886f

How to cite

You Know What I Mean: A Benchmark for Agentic Conversational Reference Grounding — arXiv:2608.29834 — arXiv (en), https://arxiv.org/abs/2608.29834 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_221befcf886f (Metraipolis archive dc_221befcf886f, sha256 dc0daf01a68295e168247df33b6528291632bc55d50795ff05b8b1e4ab91c675)
BibTeX
@misc{dc221befcf886f,
  title = {You Know What I Mean: A Benchmark for Agentic Conversational Reference Grounding — arXiv:2608.29834},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29834},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_221befcf886f; sha256 dc0daf01a68295e168247df33b6528291632bc55d50795ff05b8b1e4ab91c675},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29834. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Collaborative conversations frequently contain references whose targets are indirect rather than named: resolving "this looks like the fix discussed yesterday" requires combining conversational context with evidence from the surrounding workspace which is accessible through APIs or user interfaces. We formalize this problem as Conversational Reference Grounding (CoRG): using a given set of tools to resolve a reference in conversation to the unique external item intended by the speaker. CoRG is challenging because it combines lexical, semantic, and temporal cues distributed across the conversation and the external workspace. Agents must translate these heterogeneous signals into effective tool use: formulating strategies, discovering plausible candidates, inspecting their metadata and content, and ruling out close alternatives. We study CoRG through RepoRef, a benchmark of 400 developer-chat segments grounded in GitHub issues, pull requests, and commits across 92 repositories. Unlike single-shot retrieval tasks, RepoRef often requires multi-step tool use. Our results show that CoRG remains challenging for current agents, even the best agent reaches only 67.0% success rate, leaving one third of references unresolved. These findings position CoRG as a concrete benchmark for studying how agents search, inspect, and verify information in realistic multi-tool environments.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2608.29834 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:47:52.737Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).