Metraipolis › archive › dc_fd9dc2a02408

Compound Prompt Constraints in LLM Code Generation: A Factorial Study of Format, Persona, and Urgency — arXiv:2609.03156

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.03156
License (as archived): CC0-1.0
Archived: 2026-10-17T02:16:11.826Z
SHA-256 of the archived text: f42777d8bc4ca0c110970b6ceeba8d0ceab0aba3e259f1538c0806b01076252e
Stable URL of this page: https://metraipolis.com/documento/dc_fd9dc2a02408

How to cite

Compound Prompt Constraints in LLM Code Generation: A Factorial Study of Format, Persona, and Urgency — arXiv:2609.03156 — arXiv (en), https://arxiv.org/abs/2609.03156 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fd9dc2a02408 (Metraipolis archive dc_fd9dc2a02408, sha256 f42777d8bc4ca0c110970b6ceeba8d0ceab0aba3e259f1538c0806b01076252e)

BibTeX

@misc{dcfd9dc2a02408,
  title = {Compound Prompt Constraints in LLM Code Generation: A Factorial Study of Format, Persona, and Urgency — arXiv:2609.03156},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.03156},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fd9dc2a02408; sha256 f42777d8bc4ca0c110970b6ceeba8d0ceab0aba3e259f1538c0806b01076252e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.03156. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used in software engineering pipelines for code generation, where production prompts often combine multiple constraints. This paper presents a full-factorial empirical study of how output formatting, persona assignment, and urgency framing jointly affect LLM code-generation reliability. We evaluate all 27 combinations in a controlled 3x3x3 design and decompose each compound condition into an additive prediction and a residual interaction term that captures super-additive degradation. The study uses all 164 HumanEval+ problems across five OpenAI models from the GPT-4o family, GPT-4.1 family, and o3-mini, yielding 22,140 greedy-decoding evaluations. A format-aware extraction pipeline separates formatting failures from reasoning failures, and significance is assessed with McNemar's test, odds ratios, and 95% confidence intervals. Results show that compound constraints can produce architecture-dependent degradation not predictable from single-factor experiments. The GPT-4o family exhibits consistent super-additive effects, with pass@1 reductions 3-12 percentage points beyond additive predictions; the largest interaction is -12.2 pp on GPT-4o-mini for JSON + expert persona + moderate urgency. JSON combinations generally produce larger interactions than XML. In contrast, the GPT-4.1 family is largely resistant, while o3-mini shows a qualitatively different pattern in which structured output constraints can improve performance. These findings show that vulnerability is architecture-dependent rather than size-dependent, that individually neutral or beneficial constraints can combine to cause substantial degradation, and that compound-prompt testing should be standard in reliability assessment for LLM-assisted engineering pipelines.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-04 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.03156 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T02:03:39.999Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).