Metraipolis › archive › dc_b4ee65dd1cef
BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations — arXiv:2608.29345
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.29345
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T07:44:14.610Z
- SHA-256 of the archived text
7f410e818499bc00745eece23900c4b47f4cbf9edb4f6b9ba51d1a9874a630aa- Stable URL of this page
https://metraipolis.com/documento/dc_b4ee65dd1cef
How to cite
BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations — arXiv:2608.29345 — arXiv (en), https://arxiv.org/abs/2608.29345 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b4ee65dd1cef (Metraipolis archive dc_b4ee65dd1cef, sha256 7f410e818499bc00745eece23900c4b47f4cbf9edb4f6b9ba51d1a9874a630aa)
BibTeX
@misc{dcb4ee65dd1cef,
title = {BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations — arXiv:2608.29345},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.29345},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b4ee65dd1cef; sha256 7f410e818499bc00745eece23900c4b47f4cbf9edb4f6b9ba51d1a9874a630aa},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29345. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries implicitly rely on domain-specific knowledge, such as business logic, data conventions, and analytical practices, that is neither captured by the schema nor explicitly stated in the natural language question. Historical SQL query logs offer a valuable source of such knowledge, yet existing benchmarks do not adequately support evaluation of history-driven approaches. To address this gap, we introduce BIRD-History, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems' ability to ground underspecified natural language questions using historical SQL scripts. Each task is annotated with ground-truth labels specifying which historical queries contain relevant knowledge and which SQL clauses encode it, enabling systematic evaluation of both retrieval effectiveness and knowledge utilization. Alongside the benchmark, we propose a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, then retrieves and reranks relevant fragments for query generation. The retriever integrates seamlessly into existing few-shot text-to-SQL pipelines without requiring prompt modifications. Experiments demonstrate consistent improvements across four text-to-SQL systems, highlighting the value of leveraging historical query logs for handling underspecified queries. Dataset and code are open-sourced on this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.29345 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T07:28:26.540Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).