Metraipolis › archive › dc_e40ec023b644
OenoBench: A Wine-Domain Benchmark for Knowledge-Grounded Evaluation of Large Language Models — arXiv:2608.20106
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.20106
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T11:39:23.494Z
- SHA-256 of the archived text
007b078963d8ca0e43f5bd9a3e9e5a7495b72a8e160800d66811cee6003984db- Stable URL of this page
https://metraipolis.com/documento/dc_e40ec023b644
How to cite
OenoBench: A Wine-Domain Benchmark for Knowledge-Grounded Evaluation of Large Language Models — arXiv:2608.20106 — arXiv (en), https://arxiv.org/abs/2608.20106 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e40ec023b644 (Metraipolis archive dc_e40ec023b644, sha256 007b078963d8ca0e43f5bd9a3e9e5a7495b72a8e160800d66811cee6003984db)
BibTeX
@misc{dce40ec023b644,
title = {OenoBench: A Wine-Domain Benchmark for Knowledge-Grounded Evaluation of Large Language Models — arXiv:2608.20106},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.20106},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e40ec023b644; sha256 007b078963d8ca0e43f5bd9a3e9e5a7495b72a8e160800d66811cee6003984db},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20106. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce OenoBench, a wine-domain knowledge benchmark of 3,266 multiple-choice questions across six pillars (regions, grape varieties, viticulture, winemaking, producers, business) and four difficulty tiers. The corpus is built from 38,104 atomic, source-anchored facts extracted by 35 provenance-verified scrapers from government registries (INAO, TTB, OIV), peer-reviewed journals, and Wikipedia/Wikidata. Our methodological contribution is an LLM-driven pipeline in which language models reformat verified facts and audit the result, but never serve as the source of truth: every claim traces to a URL, every question is generated by one of five strategies across five generator families, and every question is scored by a nine-agent audit calibrated against a human gold sheet via Cohen's $\kappa$. Evaluating sixteen frontier configurations, we find: (i) overall accuracy spans 53%-84%, led by o3 at 83.6%; (ii) reasoning-mode lift concentrates in DeepSeek R1 (+6.8pp) and is absent in Claude Opus and Gemini Pro; (iii) Anthropic shows +9pp self preference on its own questions while Google shows -8pp inverse preference; (iv) frontier open-weight models share the cost-vs-accuracy Pareto frontier with proprietary reasoning models; and (v) every config gains around 33pp on closed-book solvable items, revealing a parametric-recall ceiling that only the contextual slice avoids. We release corpus, audit findings, human-review app, and construction code under CC-BY-SA-4.0. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-21 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.20106 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T11:38:27.926Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).