Metraipolis › archive › dc_c65078ae889d

BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL — arXiv:2610.00092

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.00092
License (as archived)
CC0-1.0
Archived
2026-10-06T12:07:45.682Z
SHA-256 of the archived text
f0582f1818eca2c01bb8bf22ad758c0d92b6f7321f8c6d996f661888c68a408b
Stable URL of this page
https://metraipolis.com/documento/dc_c65078ae889d

How to cite

BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL — arXiv:2610.00092 — arXiv (en), https://arxiv.org/abs/2610.00092 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c65078ae889d (Metraipolis archive dc_c65078ae889d, sha256 f0582f1818eca2c01bb8bf22ad758c0d92b6f7321f8c6d996f661888c68a408b)
BibTeX
@misc{dcc65078ae889d,
  title = {BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL — arXiv:2610.00092},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.00092},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c65078ae889d; sha256 f0582f1818eca2c01bb8bf22ad758c0d92b6f7321f8c6d996f661888c68a408b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00092. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Data agents over structured sources must fit database schema into the model's context window. Large catalogs can span many databases and thousands of columns, so cost constraints may require choosing between table coverage and serialization detail well before the context window is full. We introduce BudgetSchemaBench, an execution-grounded diagnostic for this setting. Its construction derives relevance labels mechanically from gold SQL, without human- or LLM-authored ground truth. Using a pooled 80-database catalog, we sweep four schema-context budgets and compare three representations while keeping each retriever's table ranking fixed. A source-namespace check rejects queries that obtain the correct result from the wrong database. The evaluation covers three conditions: end-to-end retrieval; frozen-gold, in which the required tables are guaranteed; and a probe that removes those tables. For the primary solver with raw serialization, raising the budget from 2.5% to 50% of the catalog improves execution accuracy on 1,279 held-out questions by 18 percentage points under lexical retrieval but only 3 under dense retrieval; the dense retriever already finds most required tables at the smallest budget. When the required tables are removed, 94.6% of correct predictions name one of them exactly, consistent with reconstruction of absent schema from parametric knowledge. For the two main solvers in the frozen-gold condition, the three representations differ by at most 2 percentage points, and the widest paired 95% confidence interval bounds the difference within +/-4 points. We observe the same qualitative patterns with one reasoning model from a different family. When retrieval is coverage-limited, execution accuracy is more sensitive to the schema budget than to the tested serializations. The diagnostic and the code used to construct and evaluate it are publicly available.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Databases (cs.DB).
Pagina del lavoro: https://arxiv.org/abs/2610.00092 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.019Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).