Metraipolis › archive › dc_47a61079efa2
RealCADBench: Benchmarking Parametric CAD Modeling from Industrial Design Intents — arXiv:2609.03773
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.03773
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T01:41:56.287Z
- SHA-256 of the archived text
74bd0aef16ef2834840b9e32419a0de456dd2113114653f1bc71d4f2185eff53- Stable URL of this page
https://metraipolis.com/documento/dc_47a61079efa2
How to cite
RealCADBench: Benchmarking Parametric CAD Modeling from Industrial Design Intents — arXiv:2609.03773 — arXiv (en), https://arxiv.org/abs/2609.03773 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_47a61079efa2 (Metraipolis archive dc_47a61079efa2, sha256 74bd0aef16ef2834840b9e32419a0de456dd2113114653f1bc71d4f2185eff53)
BibTeX
@misc{dc47a61079efa2,
title = {RealCADBench: Benchmarking Parametric CAD Modeling from Industrial Design Intents — arXiv:2609.03773},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.03773},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_47a61079efa2; sha256 74bd0aef16ef2834840b9e32419a0de456dd2113114653f1bc71d4f2185eff53},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.03773. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Parametric computer-aided design (CAD) modeling is difficult to evaluate with a single metric. Existing CAD benchmarks often emphasize synthetic or CAD-native settings, limited input modalities, or executability and IoUs alone. We introduce RealCADBench, a benchmark for intent-to-program CAD modeling from real industrial design intents. It contains 12,632 tasks from 19 factory-automation categories and spans text descriptions, 2D engineering drawings, real product pictures, and rendered images for both Part and Assembly modeling. We report results on a 1,770-task evaluation slice: 1,745 Part tasks across four input regimes and RCB-Assm25, a 25-task assembly study used in every reported assembly comparison. Each method generates FreeCAD API Python, which a shared runtime executes to export the 3D model. We evaluate the exported model using executability, Solid IoU, Surface IoU, and a rubric-based visual-semantic identity Judge. Among the nine standalone frontier large models evaluated, no model leads all four metrics. Across six frontier-scale large models, executability ranges from 0.565 to 0.812, Solid IoU from 0.2841 to 0.5379, and Surface IoU from 0.112 to 0.217 across the four Part regimes. The highest regime-balanced composite comes from a different model than the leaders on the four component metrics. On RCB-Assm25, Codex with GPT-5.5 improves executability and both IoU metrics over standalone GPT-5.5, but lowers the Judge score by 6.98 percentage points, leaving GPT-5.5 as the Judge leader. We also observe recurring failure modes, most notably missing fine structures, loss of part identity, and incorrect assembly placement. These results show that execution alone is insufficient to characterize realistic CAD modeling and that frontier models and agents differ substantially across executability, IoUs, and visual-semantic identity. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.CL). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.03773 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T01:11:43.607Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).