Metraipolis › archive › dc_86da10b97508

BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming — arXiv:2611.09933

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09933
License (as archived)
CC0-1.0
Archived
2026-11-10T18:44:22.113Z
SHA-256 of the archived text
61adffd20e55b29de5cafe94096a0dcb4e60b3721298e10cc011a1114c55a9a9
Stable URL of this page
https://metraipolis.com/documento/dc_86da10b97508

How to cite

BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming — arXiv:2611.09933 — arXiv (en), https://arxiv.org/abs/2611.09933 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_86da10b97508 (Metraipolis archive dc_86da10b97508, sha256 61adffd20e55b29de5cafe94096a0dcb4e60b3721298e10cc011a1114c55a9a9)
BibTeX
@misc{dc86da10b97508,
  title = {BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming — arXiv:2611.09933},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09933},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_86da10b97508; sha256 61adffd20e55b29de5cafe94096a0dcb4e60b3721298e10cc011a1114c55a9a9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09933. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) are emerging as natural-language interfaces for Building Information Modeling (BIM), successfully achieving information retrieval, but no benchmark exists to evaluate their ability to edit Industry Foundation Classes (IFC). We introduce BIBIMBAP, a Text-to-BIM benchmark with 100 curated tasks covering Create-Read-Update-Delete (CRUD) operations across spatial, geometric, topological, numeric, and conceptual categories. Each task includes a natural-language prompt, an IFC model, expected structured outputs, and test scripts for automated evaluation. Baseline results show limited performance (best: 50.2%) and frequent failures in constraint preservation and spatial reasoning, with some correct outputs achieved despite flawed reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.09933 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:52.417Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).