Metraipolis › archive › dc_ca0569c7381f

Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning — arXiv:2610.35083

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.35083
License (as archived)
CC0-1.0
Archived
2026-10-27T08:12:12.740Z
SHA-256 of the archived text
76ee99037a86d67e45a2a3e90ce6c8ac6ee2e63686a851387782e3cbdcdeba29
Stable URL of this page
https://metraipolis.com/documento/dc_ca0569c7381f

How to cite

Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning — arXiv:2610.35083 — arXiv (en), https://arxiv.org/abs/2610.35083 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca0569c7381f (Metraipolis archive dc_ca0569c7381f, sha256 76ee99037a86d67e45a2a3e90ce6c8ac6ee2e63686a851387782e3cbdcdeba29)
BibTeX
@misc{dcca0569c7381f,
  title = {Ask Before You Answer: A Question-Driven Evidence Reasoning Architecture (QERA) for In-Context Learning — arXiv:2610.35083},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.35083},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca0569c7381f; sha256 76ee99037a86d67e45a2a3e90ce6c8ac6ee2e63686a851387782e3cbdcdeba29},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35083. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI assistants are increasingly asked to reason over a user's own web traces: group chats, community threads, activity logs and revision histories. On CL-bench Life, a benchmark of such real-life context learning, frontier LLMs solve fewer than one task in five. We find that the request is not the specification: over 80% of the benchmark's rubrics share less than 60% of their content words with the final request, as for a rule set two turns earlier or every record that meets the user's criterion, and LLMs answering directly pass only 67% of these implicit rubrics against 81% of the others. We propose QERA, a Question-driven Evidence Reasoning Architecture that asks before it answers. QERA first turns the request into a requirement checklist and typed information needs, filtered by code for faithfulness to the user's words; it then answers from the whole conversation, asks the same questions again with quotes verified by code, and applies a guarded local repair. It needs no training or rubrics and makes about four LLM calls per task. On all 405 tasks with four frontier LLMs, QERA attains the highest solving rate on every LLM (21.0% on average, against 18.2% for the strongest of three inference-time baselines and 14.6% for direct answering) and leads every baseline significantly when pooled across LLMs, while using fewer tokens than PSCI and 4-5% of the tokens of Ctx2Skill. Ablations on GPT-6-Luna show that the gain is settled before the answer, by a fixed answer protocol (+3.8 rubric points) and the request-specific questions (+1.6), and that it is concentrated on implicit rubrics, with checking afterwards refining mainly numeric requirements.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.35083 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:56:00.250Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).