Metraipolis › archive › dc_5288323dd314

An Executable Benchmark for LLM-Based HLS Repair:Design Complexity and Repair Underconstraint — arXiv:2610.03971

Source
arXiv
Original
https://arxiv.org/abs/2610.03971
License (as archived)
CC0-1.0
Archived
2026-10-06T05:25:54.227Z
SHA-256 of the archived text
1f2470a1e5a55895158fb9103f9c6f67af30d9beb36e5815a74e342597733aa7
Stable URL of this page
https://metraipolis.com/documento/dc_5288323dd314

How to cite

An Executable Benchmark for LLM-Based HLS Repair:Design Complexity and Repair Underconstraint — arXiv:2610.03971 — arXiv, https://arxiv.org/abs/2610.03971 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5288323dd314 (Metraipolis archive dc_5288323dd314, sha256 1f2470a1e5a55895158fb9103f9c6f67af30d9beb36e5815a74e342597733aa7)
BibTeX
@misc{dc5288323dd314,
  title = {An Executable Benchmark for LLM-Based HLS Repair:Design Complexity and Repair Underconstraint — arXiv:2610.03971},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.03971},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5288323dd314; sha256 1f2470a1e5a55895158fb9103f9c6f67af30d9beb36e5815a74e342597733aa7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03971. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automated repair of High-Level Synthesis (HLS) designs using large language models (LLMs) is an emerging but underexplored problem. While LLM-based repair shows strong results on register-transfer level (RTL) Verilog, the only prior systematic study of HLS logic repair reports just 10.5% correction accuracy for GPT-4, with no analysis of why repair fails or what drives difficulty. We present the first comprehensive evaluation of LLM-based HLS repair across four models (GPT-4o, GPT-4o-mini, GPT-5.4, and Claude Opus 4.6) on 125 benchmark instances spanning eight logic bug types across three open-source HLS suites (CHStone, MachSuite, Polybench). We construct the first executable APR-style HLS repair benchmark with suite-specific functional oracles, enabling pass@k evaluation rather than the string-match approximations used in prior work. Design context and scale, rather than bug type alone, dominate repair difficulty: repair rates range from 6-45% on complex cryptographic kernels (CHStone) to 81-93% on compact algorithmic kernels (MachSuite) despite identical bug type distributions. We introduce solution multiplicity, the fraction of distinct patches generated across repeated repair attempts, as an empirical measure of repair underconstraint, and show it strongly predicts repair failure across all four models (Spearman rho = -0.393, p = 6.61 x 10^-20). Frontier models reduce underconstrained instances from 41-46 (GPT-4o, GPT-4o-mini) to just 3 (Claude Opus 4.6), with pass@1 improving from 42-53% to 73-76%. SHFT bugs remain consistently hard across all models, and semantic bug classes such as buffer indexing become reliably repairable only at frontier scale. These findings show that future APR benchmarks must include complex, executable, and weakly identifiable designs that remain challenging after frontier LLM repair.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.03971 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.764Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).