Metraipolis › archive › dc_11202f4bf4a0

Does the way we write a theory change the program an LLM builds from it? A prospective randomized study of renderer format in LLM theory-to-program translation — arXiv:2608.10314

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10314
License (as archived)
CC0-1.0
Archived
2026-10-19T06:22:12.763Z
SHA-256 of the archived text
f9eb3ba28fc4a2ae866e7e737c71b295b9f0f4cd84bb803cc4fd68221a2c65e1
Stable URL of this page
https://metraipolis.com/documento/dc_11202f4bf4a0

How to cite

Does the way we write a theory change the program an LLM builds from it? A prospective randomized study of renderer format in LLM theory-to-program translation — arXiv:2608.10314 — arXiv (en), https://arxiv.org/abs/2608.10314 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_11202f4bf4a0 (Metraipolis archive dc_11202f4bf4a0, sha256 f9eb3ba28fc4a2ae866e7e737c71b295b9f0f4cd84bb803cc4fd68221a2c65e1)
BibTeX
@misc{dc11202f4bf4a0,
  title = {Does the way we write a theory change the program an LLM builds from it? A prospective randomized study of renderer format in LLM theory-to-program translation — arXiv:2608.10314},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10314},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_11202f4bf4a0; sha256 f9eb3ba28fc4a2ae866e7e737c71b295b9f0f4cd84bb803cc4fd68221a2c65e1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10314. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A verbal theory does not run: translating it into an executable model requires choices about variables, interventions, and interactions. We tested whether the presentation of otherwise identical theoretical content systematically changes the programs produced by large language models. In a prospective preregistered randomized experiment, 16 independent assignment bits allocated 32 paired renderer slots between a structured intervention contract and connected prose. Two pinned LLM snapshots translated five anonymous theoretical accounts, yielding 320 preauthorized single-shot programs in a frozen sparse quadratic language. A deterministic evaluator measured atomic finite-difference responses (H1) and mixed interaction responses (H2). Two primary endpoints assessed cross-model matched-distance reduction and closed-set same-account identifiability, with exact randomization inference and 27 preregistered support criteria evaluated across signed-linear and magnitude-rank pipelines. Both H1 and H2 returned the registered verdict NOT_SUPPORTED; only 19 of 108 criterion evaluations passed. Same-account identifiability remained near chance (AUC 0.469-0.523, against a registered 0.80 threshold). One H2 matched-distance endpoint moved and survived multiplicity correction in the signed-linear pipeline, but the corresponding magnitude-rank result missed the registered effect-size floor, so it did not satisfy the joint support rule. Thus renderer format did not produce the uniform, family-invariant, classifiable behavioral geometry predicted in advance. The result places a concrete boundary on specification-format effects in LLM theory-to-program translation and provides a fully auditable randomized design, datasets, and software for studying executable formalization.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-12 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE) ; Computational Physics (physics.comp-ph).
Pagina del lavoro: https://arxiv.org/abs/2608.10314 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:14:48.475Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).