Metraipolis › archive › dc_69ea08a27ead
Large Language Models and their Awareness of Mechanics and Spatial Geometry — arXiv:2608.14615
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14615
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:05:35.956Z
- SHA-256 of the archived text
af540db050a4047ca3dd8568150403cf887901b521e0d84da0a4582a614ae671- Stable URL of this page
https://metraipolis.com/documento/dc_69ea08a27ead
How to cite
Large Language Models and their Awareness of Mechanics and Spatial Geometry — arXiv:2608.14615 — arXiv (en), https://arxiv.org/abs/2608.14615 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_69ea08a27ead (Metraipolis archive dc_69ea08a27ead, sha256 af540db050a4047ca3dd8568150403cf887901b521e0d84da0a4582a614ae671)
BibTeX
@misc{dc69ea08a27ead,
title = {Large Language Models and their Awareness of Mechanics and Spatial Geometry — arXiv:2608.14615},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14615},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_69ea08a27ead; sha256 af540db050a4047ca3dd8568150403cf887901b521e0d84da0a4582a614ae671},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14615. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) perform well on established code-generation and mathematical-reasoning benchmarks, but their capabilities in mechanics and spatial geometry, here denoted as mechanical engineering awareness, has not been quantified systematically. We present MecEng, a fully automated benchmark that evaluates LLMs on the creation of multibody simulation models from parameterized textual descriptions. The benchmark comprises 84 generic tasks on three difficulty levels, ranging from rigid-body systems with joints and contact to flexible multibody systems that require exact 3D geometry generation, tetrahedral finite-element meshing, and Hurty-Craig-Bampton model order reduction of machine parts. A dedicated pipeline with LLMs generates simulation-ready geometry from text using Netgen, and builds multibody system models for the code Exudyn, which are then verified against expert ground truth on several levels: system-graph isomorphism including graph node annotations, numerical solutions, and part-specific measures such as mass, geometry, and eigenfrequencies. In total, 32 open-weight and two proprietary LLMs are evaluated. On rigid-body tasks, the best open-weight model obtains an overall success rate of 86.0%, compared to 91.4% for the strongest proprietary model, while flexible multibody tasks remain considerably harder. Additional studies quantify the influence of sampling temperature, reasoning, prompt design, model size, and LLM-release date. The results indicate rapidly improving, but still error-prone, mechanical engineering awareness of current LLMs. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.14615 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.714Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).