Metraipolis › archive › dc_8b4aed149b5d
Trajectory-Derived Confidence for Reliable, Resource-Aware Clinical Text-to-SQL Agents — arXiv:2610.04156
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.04156
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:26:01.353Z
- SHA-256 of the archived text
c19c3d2f5ae02175c7a76130b9e4d161ce5bf6de08963249298c972ac6efafec- Stable URL of this page
https://metraipolis.com/documento/dc_8b4aed149b5d
How to cite
Trajectory-Derived Confidence for Reliable, Resource-Aware Clinical Text-to-SQL Agents — arXiv:2610.04156 — arXiv, https://arxiv.org/abs/2610.04156 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8b4aed149b5d (Metraipolis archive dc_8b4aed149b5d, sha256 c19c3d2f5ae02175c7a76130b9e4d161ce5bf6de08963249298c972ac6efafec)
BibTeX
@misc{dc8b4aed149b5d,
title = {Trajectory-Derived Confidence for Reliable, Resource-Aware Clinical Text-to-SQL Agents — arXiv:2610.04156},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.04156},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8b4aed149b5d; sha256 c19c3d2f5ae02175c7a76130b9e4d161ce5bf6de08963249298c972ac6efafec},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04156. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM agents for clinical text-to-SQL applications reason autonomously over multiple steps but cannot assess whether their own reasoning or outputs can be trusted. In high leverage applications such as healthcare, this presents a critical risk where system mistakes can be costly. These reliability failures are also resource failures: an incorrect reasoning trajectory spends computation budget on outputs that must be discarded. We introduce Sentinel, a trajectory-derived, classifier-based confidence layer that analyzes an agent's reasoning, code and database outputs to decide at three points whether to stop: refusing unanswerable questions before the agent runs, halting doomed trajectories mid-run, and withholding untrustworthy answers at delivery. Here, utilizing Chow's rule, we optimize decisions under the EHRSQL shared task's Reliability Score, which penalizes incorrect answers given a utility weighting, and find on the benchmark EHRSQL that Sentinel raises this score from +0.08 to +0.24 when mistakes have a low utility weighting, well above the +0.03 earned by refusing every question, with delivered-answer accuracy rising from 54% to 69% as coverage falls from 84% to 55%. At higher stakes the agents we test rarely answer reliably enough to deliver, and Sentinel detects this on its own, abstaining to that same +0.03 where the unmonitored agent scores -3.02. The same stopping decisions cut computation: at low stakes, where the system still answers, Sentinel eliminates 13-28% of agent steps at little or no reliability cost. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.04156 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:05.801Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).