Metraipolis › archive › dc_ab5eee41fdd5

PLE: Physical-Engineering's Last Exam — arXiv:2610.30537

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.30537
License (as archived)
CC0-1.0
Archived
2026-10-23T15:12:54.258Z
SHA-256 of the archived text
50482001305f564e837166815f9f88427764e95c79c1d82a22daf11764234c3f
Stable URL of this page
https://metraipolis.com/documento/dc_ab5eee41fdd5

How to cite

PLE: Physical-Engineering's Last Exam — arXiv:2610.30537 — arXiv (en), https://arxiv.org/abs/2610.30537 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ab5eee41fdd5 (Metraipolis archive dc_ab5eee41fdd5, sha256 50482001305f564e837166815f9f88427764e95c79c1d82a22daf11764234c3f)
BibTeX
@misc{dcab5eee41fdd5,
  title = {PLE: Physical-Engineering's Last Exam — arXiv:2610.30537},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.30537},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ab5eee41fdd5; sha256 50482001305f564e837166815f9f88427764e95c79c1d82a22daf11764234c3f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30537. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Frontier language models now score above 90% on popular knowledge benchmarks and are increasingly deployed as agents that carry out multi-hour professional work. Yet no existing benchmark tests whether they can perform the real-world, economically valuable work of a professional physical engineer, such as modelling physics and legal code requirements across a multi-format project corpus, optimising material and installation costs, and issuing 2D/3D engineering design packages that a licensed reviewer would sign off on. Existing engineering benchmarks focus on textbook question answering, isolated tool calls or self-contained toy design problems. As a result, most of these benchmarks are already saturated by leading agents and solving them is far away from delivering real engineering design. We introduce PLE, a benchmark of 85 long-horizon, agentic tasks in professional physical engineering consisting of electrical, civil, structural and mechanical engineering, where solving a task means completing a realistic, billable engineering deliverable. PLE combines two complementary curation paths. Expert-Authored tasks are created end to end by practising engineers from a professional data provider's expert pool. Co-Authored tasks start from problem statements written by practising engineers; an adversarial Creator-Solver agent pipeline constructs each task's realistic project environment, executable grader and reference deliverables before sending them to expert engineers for validation and difficulty intensification. We evaluate a variety of frontier LLMs under seven publicly available agent harnesses. Even the strongest configuration reaches a mean score of only 50.6%, while spending 27M tokens and 63 minutes per task on average, demonstrating the substantial gap between current agents and autonomous physical engineering design.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.30537 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-23T15:09:08.372Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).