Metraipolis › archive › dc_024a3d60ecf9

LLM-Based FORM Code Generation with Verification-Driven Fine-Tuning — arXiv:2609.23367

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.23367
License (as archived)
CC0-1.0
Archived
2026-10-09T06:50:33.760Z
SHA-256 of the archived text
b924ef9fbdf718fc416ff9c68457f4c90d2a1f4f0824f0231f47e57c62f622ac
Stable URL of this page
https://metraipolis.com/documento/dc_024a3d60ecf9

How to cite

LLM-Based FORM Code Generation with Verification-Driven Fine-Tuning — arXiv:2609.23367 — arXiv (en), https://arxiv.org/abs/2609.23367 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_024a3d60ecf9 (Metraipolis archive dc_024a3d60ecf9, sha256 b924ef9fbdf718fc416ff9c68457f4c90d2a1f4f0824f0231f47e57c62f622ac)
BibTeX
@misc{dc024a3d60ecf9,
  title = {LLM-Based FORM Code Generation with Verification-Driven Fine-Tuning — arXiv:2609.23367},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.23367},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_024a3d60ecf9; sha256 b924ef9fbdf718fc416ff9c68457f4c90d2a1f4f0824f0231f47e57c62f622ac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23367. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

FORM is a domain-specific symbolic manipulation language widely used in particle physics for processing the very large algebraic expressions arising from multi-loop Feynman diagram calculations. Despite its central role in precision theoretical physics, no artificial-intelligence tooling exists, to our knowledge, for assisting physicists in writing FORM code. We show that contemporary large language models (LLMs), including frontier models with hundreds of billions of parameters, achieve a zero-percent execution pass rate on our instruction-following and tutorial-style FORM tasks without documentation in a single attempt, establishing FORM as a genuine zero-shot language for LLMs at the time of writing. We then present a verification-driven data generation pipeline that uses the FORM binary itself as an execution oracle to produce and validate a corpus of 4,633 training examples spanning deterministic computations, open-ended programs, tutorial code, and knowledge question-answer pairs. Fine-tuning a compact open-weights model (Qwen3-8B) with quantized low-rank adaptation (QLoRA) yields a specialist that, evaluated on four complementary benchmarks (840 tasks, single attempt each), decisively outperforms frontier models with up to 756B parameters in execution rate and in strict, FORM-verified output matching on the larger benchmarks, and remains statistically indistinguishable from them on the smaller, harder ones. General reasoning and coding capabilities are preserved within 2.6 percentage points.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: High Energy Physics - Phenomenology (hep-ph) ; Computational Engineering, Finance, and Science (cs.CE); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.23367 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:45.118Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).