Metraipolis › archive › dc_025e7bcdf4a7
Jev-as-a-Judge for RAG Claim Verification: Confidence-Gated Cascading Between a Non-Generative Verifier and a Frontier LLM — arXiv:2610.20634
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.20634
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:52:02.146Z
- SHA-256 of the archived text
0374f16c1f881b9b13deaffec697e0e154cdb89e049b507f3e2c8444f985f3c7- Stable URL of this page
https://metraipolis.com/documento/dc_025e7bcdf4a7
How to cite
Jev-as-a-Judge for RAG Claim Verification: Confidence-Gated Cascading Between a Non-Generative Verifier and a Frontier LLM — arXiv:2610.20634 — arXiv (en), https://arxiv.org/abs/2610.20634 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_025e7bcdf4a7 (Metraipolis archive dc_025e7bcdf4a7, sha256 0374f16c1f881b9b13deaffec697e0e154cdb89e049b507f3e2c8444f985f3c7)
BibTeX
@misc{dc025e7bcdf4a7,
title = {Jev-as-a-Judge for RAG Claim Verification: Confidence-Gated Cascading Between a Non-Generative Verifier and a Frontier LLM — arXiv:2610.20634},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.20634},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_025e7bcdf4a7; sha256 0374f16c1f881b9b13deaffec697e0e154cdb89e049b507f3e2c8444f985f3c7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20634. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Retrieval-augmented generation asks one question at volume: does this evidence support this claim? The answer is one bit, yet most pipelines obtain it from a text generator and pay generation prices for the prose wrapped around it. We compare a non-generative decision model, which returns a typed value from a declared enumeration together with a native probability and no text, against a frontier LLM configured as an engineer would deploy it for binary classification. Both answer the same 495 human-labelled claims from LLM-AggreFact, so every comparison is within-item. We report three things. First, the two systems are statistically indistinguishable on the metric this task is conventionally scored by, a paired difference of -0.56 points, Jev minus Astra, at a 187-fold cost difference. Second, that tie carries almost no information about deployment: the systems disagree on 15.2% of claims, and the cheaper one admits 23.2% of the claims annotators marked unsupported against the frontier model's 14.1%. A component whose job is to catch unsupported claims should be judged on that rate, and the conventional headline hides it. Third, because the errors differ they compose: routing the claims the cheap model is confident about and escalating the rest reaches 74.10 at USD 2.92 per thousand claims against 73.83 at USD 8.93, with the threshold chosen out of fold. But a matched random-escalation control at in-sample thresholds, spending the same budget by coin flip, does as well, so what buys the saving is the escalation share and not the probability that picks what to escalate. Measured against the cheap model it is built from, the cascade costs 61-fold more for 0.83 points, 95% CI [-3.36, 4.86], and it is the more permissive guard of the two. Equal accuracy conceals unequal errors. All predictions, prompts, pricing and code are released. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.20634 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:47.600Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).