Metraipolis › archive › dc_8b4aed149b5d

Trajectory-Derived Confidence for Reliable, Resource-Aware Clinical Text-to-SQL Agents — arXiv:2610.04156

Source
arXiv
Original
https://arxiv.org/abs/2610.04156
License (as archived)
CC0-1.0
Archived
2026-10-06T05:26:01.353Z
SHA-256 of the archived text
c19c3d2f5ae02175c7a76130b9e4d161ce5bf6de08963249298c972ac6efafec
Stable URL of this page
https://metraipolis.com/documento/dc_8b4aed149b5d

How to cite

Trajectory-Derived Confidence for Reliable, Resource-Aware Clinical Text-to-SQL Agents — arXiv:2610.04156 — arXiv, https://arxiv.org/abs/2610.04156 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8b4aed149b5d (Metraipolis archive dc_8b4aed149b5d, sha256 c19c3d2f5ae02175c7a76130b9e4d161ce5bf6de08963249298c972ac6efafec)
BibTeX
@misc{dc8b4aed149b5d,
  title = {Trajectory-Derived Confidence for Reliable, Resource-Aware Clinical Text-to-SQL Agents — arXiv:2610.04156},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04156},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8b4aed149b5d; sha256 c19c3d2f5ae02175c7a76130b9e4d161ce5bf6de08963249298c972ac6efafec},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04156. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM agents for clinical text-to-SQL applications reason autonomously over multiple steps but cannot assess whether their own reasoning or outputs can be trusted. In high leverage applications such as healthcare, this presents a critical risk where system mistakes can be costly. These reliability failures are also resource failures: an incorrect reasoning trajectory spends computation budget on outputs that must be discarded. We introduce Sentinel, a trajectory-derived, classifier-based confidence layer that analyzes an agent's reasoning, code and database outputs to decide at three points whether to stop: refusing unanswerable questions before the agent runs, halting doomed trajectories mid-run, and withholding untrustworthy answers at delivery. Here, utilizing Chow's rule, we optimize decisions under the EHRSQL shared task's Reliability Score, which penalizes incorrect answers given a utility weighting, and find on the benchmark EHRSQL that Sentinel raises this score from +0.08 to +0.24 when mistakes have a low utility weighting, well above the +0.03 earned by refusing every question, with delivered-answer accuracy rising from 54% to 69% as coverage falls from 84% to 55%. At higher stakes the agents we test rarely answer reliably enough to deliver, and Sentinel detects this on its own, abstaining to that same +0.03 where the unmonitored agent scores -3.02. The same stopping decisions cut computation: at low stakes, where the system still answers, Sentinel eliminates 13-28% of agent steps at little or no reliability cost.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.04156 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.801Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).