Metraipolis › archive › dc_3f06290d474d
The Complexity Kink: A Prompt-Side Structural Complexity Index for Code-Generation Reliability — arXiv:2609.19616
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19616
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T04:08:41.041Z
- SHA-256 of the archived text
1bbd192cfb58bd3e0ba0ed60fdc73a480773a8cf45e24ff751947748bdb0ad34- Stable URL of this page
https://metraipolis.com/documento/dc_3f06290d474d
How to cite
The Complexity Kink: A Prompt-Side Structural Complexity Index for Code-Generation Reliability — arXiv:2609.19616 — arXiv (en), https://arxiv.org/abs/2609.19616 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3f06290d474d (Metraipolis archive dc_3f06290d474d, sha256 1bbd192cfb58bd3e0ba0ed60fdc73a480773a8cf45e24ff751947748bdb0ad34)
BibTeX
@misc{dc3f06290d474d,
title = {The Complexity Kink: A Prompt-Side Structural Complexity Index for Code-Generation Reliability — arXiv:2609.19616},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19616},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3f06290d474d; sha256 1bbd192cfb58bd3e0ba0ed60fdc73a480773a8cf45e24ff751947748bdb0ad34},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19616. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Complexity measured from generated code is failure-dependent: a difficult prompt can yield a short failing program and be assigned low output complexity. We introduce a six-dimension prompt-side structural-complexity index scored before generation and kept separate from correctness. We select 5,000 Python prompts across six bands of a preliminary single-rater rubric. Four out-of-panel LLM raters rescore the locked prompts, giving 19,997 score rows; composite inter-rater reliability is ICC = 0.872 on the 4,998 prompts with all four ratings. We evaluate 21 models per prompt, yielding 105,000 generations. In the unadjusted mean-pooled analysis, pass rate has a nonmonotone breakpoint at composite 13.75, with 79.9% at or below and 87.6% above. This is not a universal failure cutoff. Task-type fixed effects shift the breakpoint to 10.75 and cut the regime gap from 7.6 to 2.1 points. A construction-frame control shifts it to 8.50 with a raw gap of -3.5 points, and neither frame alone reproduces the pooled +7.6-point change. Model-specific fits include 16 upward and five downward changes. A 365-prompt audit-clean extension matches the original five-model estimates at bins 15 and 16 but adds only 14 prompts above bin 16. Among zero-pass generations with computable Lizard complexity, 28.5% pair a prompt composite above 8 with output complexity at most 10. Human agreement is moderate and rater-dependent on a disagreement-enriched calibration set; paraphrase and cross-language rescoring preserve score ordering. Overidentification tests reject the joint restrictions on the six dimensions, so we treat the composite as an index and make no causal interpretation of the 2SLS estimates. The contribution is a pre-generation measurement framework and a bounded observational analysis of reliability regimes. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.19616 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:48:18.270Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).