Metraipolis › archive › dc_afd88127fa4d
Automatic Harness Evolution for Hardware Design Verification: Can LLMs Consolidate Gains Across Discovered Harnesses? — arXiv:2609.28908
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.28908
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T05:26:24.903Z
- SHA-256 of the archived text
1f8f999495b75e7fcc56ca179fdfbbeb500c453750148409533077f85d038668- Stable URL of this page
https://metraipolis.com/documento/dc_afd88127fa4d
How to cite
Automatic Harness Evolution for Hardware Design Verification: Can LLMs Consolidate Gains Across Discovered Harnesses? — arXiv:2609.28908 — arXiv (en), https://arxiv.org/abs/2609.28908 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_afd88127fa4d (Metraipolis archive dc_afd88127fa4d, sha256 1f8f999495b75e7fcc56ca179fdfbbeb500c453750148409533077f85d038668)
BibTeX
@misc{dcafd88127fa4d,
title = {Automatic Harness Evolution for Hardware Design Verification: Can LLMs Consolidate Gains Across Discovered Harnesses? — arXiv:2609.28908},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.28908},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_afd88127fa4d; sha256 1f8f999495b75e7fcc56ca179fdfbbeb500c453750148409533077f85d038668},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28908. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agent behavior depends on the harness surrounding a language model, but it remains unclear whether language models can reliably improve such harnesses for hardware-design tasks. We study automatic harness evolution around a fixed subject model on 12 proprietary design-verification root-cause localization tasks. Across five trials per task, automatically evolved harnesses increased completed attempts by 71-76% and any-hit task coverage by 80-100%, while total correct attempts improved by only 18-24%. The strongest success reproducible at least twice result improved by one task, and later candidates exchanged gains across tasks rather than preserving them. An auxiliary candidate improved on a four-task validation set excluded from search but tied its baseline on a subsequent 12-task replay containing both search and validation tasks, so the selected gain did not persist across the full pool. Across the tested lineage, useful search, evidence, and finalization behaviors appeared in different candidates but did not consistently consolidate into a single harness that dominated across tasks and metrics. In a separate CVDP cross-benchmark case study, an automatically evolved defined-width repair harness produced 35.6% more functional passes than its 142-task reference baseline; the final functional verifier scored completed outputs but was not shown to the subject agent during repair. These results support archive-aware selection when evolution yields complementary specializations without consistent consolidation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.LG). Categorie arXiv: Software Engineering (cs.SE) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.28908 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T04:50:39.566Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).