Metraipolis › archive › dc_b7688a67a3e2

Validation and Simulation Catch Different Errors: Four Levels of Evaluation for LLM-Generated Circuits — arXiv:2609.26830

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.26830
License (as archived): CC0-1.0
Archived: 2026-10-08T09:11:50.509Z
SHA-256 of the archived text: 1a273f41d4c0d680dd2a5c0462f0fc942e4f852fed6f2931cf7ae43ee96d2e95
Stable URL of this page: https://metraipolis.com/documento/dc_b7688a67a3e2

How to cite

Validation and Simulation Catch Different Errors: Four Levels of Evaluation for LLM-Generated Circuits — arXiv:2609.26830 — arXiv (en), https://arxiv.org/abs/2609.26830 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b7688a67a3e2 (Metraipolis archive dc_b7688a67a3e2, sha256 1a273f41d4c0d680dd2a5c0462f0fc942e4f852fed6f2931cf7ae43ee96d2e95)

BibTeX

@misc{dcb7688a67a3e2,
  title = {Validation and Simulation Catch Different Errors: Four Levels of Evaluation for LLM-Generated Circuits — arXiv:2609.26830},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.26830},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b7688a67a3e2; sha256 1a273f41d4c0d680dd2a5c0462f0fc942e4f852fed6f2931cf7ae43ee96d2e95},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26830. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Simulation success is not equivalent to structural correctness for LLM-generated circuits. We define and measure four evaluation levels -- schema validity, topological validity, backend executability, and component-set agreement -- on a 150-circuit trilingual benchmark, through a deployed pipeline built on a typed circuit interchange representation. The levels are not nested. On gpt-4o-mini, 16 of 150 circuits (10.7%, 95% CI 6.7-16.6) were rejected by the topological validator but executed in ngspice with no error or warning; 12 of these contained exactly the requested components, with one terminal disconnected. Conversely, 7 circuits (4.7%) passed the validator and ngspice refused them. Ten failed both checks and 117 passed both, so each check detects a class the other misses. A minimal three-component divider shows the cost: a dangling resistor reports 5.00 V instead of 2.50 V while ngspice stays silent. A paired ablation, in which every arm is evaluated from the same model sample rather than a fresh one, separates each repair stage from sampling noise. On a stratified 45-circuit subsample, model repair raised topological validity from 40.0% to 84.4% (+20 circuits, no regressions) while moving executability by a net 6 (+7, -1), an effect this sample size does not resolve, and component agreement by 2. One circuit moved in opposite directions at two levels in a single repair step. Against a direct-netlist baseline the pipeline executed 88.7% against 47.3%, or 62.7% under an accounting that credits the baseline with every failure we cannot confidently attribute to the netlist. These results support a narrow methodological conclusion: structural validation and simulation should be reported as distinct evaluation stages for LLM-generated circuits. A circuit that runs is not necessarily structurally valid, and a structurally valid circuit is not necessarily executable.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI).
Categorie arXiv: Hardware Architecture (cs.AR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.26830 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:13:26.573Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).