Metraipolis › archive › dc_7dfed76d564e

Can LLMs in Draft-Verify-Revise Pipelines Resolve Deictic Ambiguity? — arXiv:2609.12162

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12162
License (as archived)
CC0-1.0
Archived
2026-10-16T03:02:35.731Z
SHA-256 of the archived text
d7e450f8ba6e5e7736d3abf38022b16c3bb5abf6b84ef5a63cb4e283da551120
Stable URL of this page
https://metraipolis.com/documento/dc_7dfed76d564e

How to cite

Can LLMs in Draft-Verify-Revise Pipelines Resolve Deictic Ambiguity? — arXiv:2609.12162 — arXiv (en), https://arxiv.org/abs/2609.12162 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7dfed76d564e (Metraipolis archive dc_7dfed76d564e, sha256 d7e450f8ba6e5e7736d3abf38022b16c3bb5abf6b84ef5a63cb4e283da551120)
BibTeX
@misc{dc7dfed76d564e,
  title = {Can LLMs in Draft-Verify-Revise Pipelines Resolve Deictic Ambiguity? — arXiv:2609.12162},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12162},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7dfed76d564e; sha256 d7e450f8ba6e5e7736d3abf38022b16c3bb5abf6b84ef5a63cb4e283da551120},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12162. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Draft-verify-revise is a common LLM orchestration pattern for scaling inference-time compute. One LLM drafts, a second critiques the draft and provides feedback, and a third uses that feedback to revise the draft into the final output. As context cascades between stages, LLMs at different stages can resolve a context-dependent expression such as "previous" differently. When that happens, the expression undergoes a deictic shift, a change in what it refers to. This phenomenon was studied with a synthetic dataset of 10 base examples, each rendered in three conditions. Holding the shared components constant, the conditions varied whether the draft stage LLM (the assistant) or the verify stage LLM (the grader) resolved the expression correctly, and how much independent reasoning the revise stage LLM (the meta-evaluator) needed to determine which reading was correct. Six models from three providers were tested across 21 reasoning effort configurations using e-values for sequential testing, in a primary experiment and an ablation experiment that removed error classification labels from the grader's feedback. A separate LLM analyzed the meta-evaluator's stated rationale for each wrong verdict. Balanced accuracy (the unweighted mean of sensitivity and specificity) ranged from 0.156, below chance, to near-perfect. GPT-5.2 rose from 0.156 without reasoning to 0.942 at its highest reasoning effort level, while Gemini 3 Pro stayed above 0.94 at every level. Gemini 3 Pro at low reasoning effort outscored GPT-5.2 at xhigh reasoning effort for roughly 5% of the cost per trial. When the meta-evaluator erred, it tended to rely on surface cues rather than operational reasoning. Context engineers implementing draft-verify-revise pipelines should be wary of deictic shifts and make the intended referent explicit at each stage.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.12162 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T02:51:27.760Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).