Metraipolis › archive › dc_da38ced638e8
VisCodeBench: a new benchmark dataset for Text-to-Vis reflecting real-world practice — arXiv:2611.02353
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02353
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:23:38.567Z
- SHA-256 of the archived text
fc7ea4a6c5147d773e965169ebc05ce8a0de5b6abd7044d1459bbdf9f9afa280- Stable URL of this page
https://metraipolis.com/documento/dc_da38ced638e8
How to cite
VisCodeBench: a new benchmark dataset for Text-to-Vis reflecting real-world practice — arXiv:2611.02353 — arXiv (en), https://arxiv.org/abs/2611.02353 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da38ced638e8 (Metraipolis archive dc_da38ced638e8, sha256 fc7ea4a6c5147d773e965169ebc05ce8a0de5b6abd7044d1459bbdf9f9afa280)
BibTeX
@misc{dcda38ced638e8,
title = {VisCodeBench: a new benchmark dataset for Text-to-Vis reflecting real-world practice — arXiv:2611.02353},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02353},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da38ced638e8; sha256 fc7ea4a6c5147d773e965169ebc05ce8a0de5b6abd7044d1459bbdf9f9afa280},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02353. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Generating visualisations from natural language requests (text-to-vis) enables users to explore data more efficiently. While LLMs can complete simple visualisation requests, their performance on challenging ones is not well covered by existing benchmarks. We introduce VisCodeBench, an end-to-end text-to-vis benchmark with 2,672 samples from three different domains: educational, scientific, and general audience scenarios with varying levels of complexity. We benchmark three visualisation systems---Codex, Claude Code, and LIDA---paired with a range of open and commercial LLMs across seven metrics covering execution, chart content, and visual quality. While basic execution is near-solved for strong systems, no system comes close to reproducing the reference data, with the best displayed data similarity reaching only $0.81$ and falling to $0.69$ on our hardest subset for this metric. We further find that frontier models separate on chart correctness and efficiency, with ClaudeCode-Opus-4.7 improving chart correctness by around 5 points over Codex-GPT-5 while running roughly 20 seconds faster. Separately, we find that system design matters as much as the underlying model: despite both running on GPT-5, Codex clearly outperforms LIDA, showing that the surrounding prompting, execution, and refinement pipeline can shape outcomes as much as the model itself. Finally, latency remains a practical obstacle, as even the strongest systems take 39-78 seconds to produce results. VisCodeBench is available at \href{ this https URL }{ this https URL } and accessed via a leaderboard; the full dataset is kept private to prevent LLM training use.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2611.02353 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:07.963Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).