Metraipolis › archive › dc_502fc65fc702

PLCWorld: Benchmarking LLM-Generated PLC Programs in Closed-Loop Plant Simulation — arXiv:2610.02982

Source: arXiv
Original: https://arxiv.org/abs/2610.02982
License (as archived): CC0-1.0
Archived: 2026-10-05T22:49:26.325Z
SHA-256 of the archived text: 039f44f837311303a4c7e73f4213baad956a614fd8842835e5fe679f459a3065
Stable URL of this page: https://metraipolis.com/documento/dc_502fc65fc702

How to cite

PLCWorld: Benchmarking LLM-Generated PLC Programs in Closed-Loop Plant Simulation — arXiv:2610.02982 — arXiv, https://arxiv.org/abs/2610.02982 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_502fc65fc702 (Metraipolis archive dc_502fc65fc702, sha256 039f44f837311303a4c7e73f4213baad956a614fd8842835e5fe679f459a3065)

BibTeX

@misc{dc502fc65fc702,
  title = {PLCWorld: Benchmarking LLM-Generated PLC Programs in Closed-Loop Plant Simulation — arXiv:2610.02982},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02982},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_502fc65fc702; sha256 039f44f837311303a4c7e73f4213baad956a614fd8842835e5fe679f459a3065},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02982. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Programmable logic controllers (PLCs) coordinate industrial equipment by reading sensor inputs and issuing control commands. Evaluating whether large language model (LLM)-generated PLC programs satisfy task requirements and safety constraints requires observing how their commands affect device and workpiece states. We introduce PLCWorld, a common closed-loop execution environment and benchmark that couples Structured Text (ST) execution with simulated plant responses and sensor feedback. Grounded in control relations identified in industrial PLC programs and engineering documentation, PLCWorld contains 100 synthetic tasks and 473 registered task-condition pairs across Motion Control and Material Handling, with difficulty defined by control-dependency scope. A common protocol reports Task Success and Safety Violation separately. Validation combines practitioner review, reference and alternative programs, targeted counterexamples, specification-evaluator alignment checks, and comparisons with independent ST runtimes. Reference and alternative programs satisfy their applicable cases, while all 542 targeted counterexamples activate their designated evaluator rules under at least one registered condition. Execution Gap relates submission-profile acceptance to subsequent task failure or observed Safety Violation. Across the constructed task groups, direct GPT-5.5 achieves 82.70% Task Success on Easy cases but 25.10% on Hard cases. Evaluations of six LLMs and four adapted generation-and-verification workflows further expose differences between completion, safety, and generation cost. Our code, simulation environment, benchmark tasks, and baseline implementations are publicly available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.02982 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.664Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).