Metraipolis › archive › dc_2de5cc5ba4cb

Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation — arXiv:2608.02672

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02672
License (as archived)
CC0-1.0
Archived
2026-10-31T12:17:36.218Z
SHA-256 of the archived text
be7ef08194bb32ea79e9098224c54742504b42eb0eb232d827ac04b402825e69
Stable URL of this page
https://metraipolis.com/documento/dc_2de5cc5ba4cb

How to cite

Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation — arXiv:2608.02672 — arXiv (en), https://arxiv.org/abs/2608.02672 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2de5cc5ba4cb (Metraipolis archive dc_2de5cc5ba4cb, sha256 be7ef08194bb32ea79e9098224c54742504b42eb0eb232d827ac04b402825e69)
BibTeX
@misc{dc2de5cc5ba4cb,
  title = {Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation — arXiv:2608.02672},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02672},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2de5cc5ba4cb; sha256 be7ef08194bb32ea79e9098224c54742504b42eb0eb232d827ac04b402825e69},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02672. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Cloud misconfiguration remains a leading cause of security incidents, yet whether LLMs and SLMs can generate security-compliant Infrastructure-as-Code is an open question. We benchmark seven models, three closed LLMs (Claude Opus 4, GPT-5.4, Gemini 2.5 Pro) and four open SLMs (Qwen2.5-Coder-14B, WizardCoder-33B, CodeLlama-13B, Magicoder-S-CL-7B), on AWS Terraform generation across 17 scenarios, integrating Checkov and Trivy scanners into a GitLab CI/CD pipeline and evaluating two prompt strategies at three security levels (pass@5). Syntactic validity and security compliance are largely orthogonal properties in LLM-generated IaC, a model that reliably produces well-formed Terraform does not necessarily produce secure Terraform: WizardCoder-33B achieves 77.8% validate rate yet zero Checkov compliance, while Claude Opus 4 reaches 23.1% Checkov and 92.5% Trivy pass rates under detailed security prompting. Consequently, prompt engineering alone is insufficient: automated multi-tool scanning remains a necessary complement to LLM-assisted IaC generation regardless of model family or prompt strategy. All artifacts are publicly available.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.AI).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.02672 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:05:52.923Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).