Metraipolis › archive › dc_86da10b97508
BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming — arXiv:2611.09933
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09933
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:44:22.113Z
- SHA-256 of the archived text
61adffd20e55b29de5cafe94096a0dcb4e60b3721298e10cc011a1114c55a9a9- Stable URL of this page
https://metraipolis.com/documento/dc_86da10b97508
How to cite
BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming — arXiv:2611.09933 — arXiv (en), https://arxiv.org/abs/2611.09933 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_86da10b97508 (Metraipolis archive dc_86da10b97508, sha256 61adffd20e55b29de5cafe94096a0dcb4e60b3721298e10cc011a1114c55a9a9)
BibTeX
@misc{dc86da10b97508,
title = {BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming — arXiv:2611.09933},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09933},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_86da10b97508; sha256 61adffd20e55b29de5cafe94096a0dcb4e60b3721298e10cc011a1114c55a9a9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09933. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) are emerging as natural-language interfaces for Building Information Modeling (BIM), successfully achieving information retrieval, but no benchmark exists to evaluate their ability to edit Industry Foundation Classes (IFC). We introduce BIBIMBAP, a Text-to-BIM benchmark with 100 curated tasks covering Create-Read-Update-Delete (CRUD) operations across spatial, geometric, topological, numeric, and conceptual categories. Each task includes a natural-language prompt, an IFC model, expected structured outputs, and test scripts for automated evaluation. Baseline results show limited performance (best: 50.2%) and frequent failures in constraint preservation and spatial reasoning, with some correct outputs achieved despite flawed reasoning. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.09933 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:37:52.417Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).