Metraipolis › archive › dc_7343e804a905
Identity or Prompt Noise? A Calibrated Invariance Audit of LLM Code Generation — arXiv:2609.22511
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22511
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T07:13:51.962Z
- SHA-256 of the archived text
ebc34c01564d6f853effbaa41b0277b40944e47473296a1e1875712c0fc91543- Stable URL of this page
https://metraipolis.com/documento/dc_7343e804a905
How to cite
Identity or Prompt Noise? A Calibrated Invariance Audit of LLM Code Generation — arXiv:2609.22511 — arXiv (en), https://arxiv.org/abs/2609.22511 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7343e804a905 (Metraipolis archive dc_7343e804a905, sha256 ebc34c01564d6f853effbaa41b0277b40944e47473296a1e1875712c0fc91543)
BibTeX
@misc{dc7343e804a905,
title = {Identity or Prompt Noise? A Calibrated Invariance Audit of LLM Code Generation — arXiv:2609.22511},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22511},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7343e804a905; sha256 ebc34c01564d6f853effbaa41b0277b40944e47473296a1e1875712c0fc91543},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22511. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Identity cues are irrelevant to a fixed programming specification, but raw counterfactual differences can arise from unequal samples and prompt wording. We audit 30.73 million executed Python generations from seven checkpoints on HumanEval+ and MBPP+, supplemented by an exploratory 550B slice, under model-assigned gender, country, and occupation personas. Within-task randomization and false-discovery-rate control identify occupation as the most consistent structural axis: CodeBLEU dispersion exceeds its exchangeability null in 10/14 model--benchmark cells, remains significant in 8/12 full-coverage cells, and exceeds the country ratio in every paired cell, although the median excess is only 0.141 points. In six high-pass-rate cells, occupation dispersion replicates across token similarity, length, comments, reference similarity, and complexity, while pass-rate dispersion is significant in none. Country leads raw dispersion in 12/14 cells but has a median calibrated ratio of 1.00. Gender-associated variation cannot be separated from persona wording in this design. Thus, the evidence supports small, reproducible occupation-conditioned changes in code form, not stable disadvantage to named identities or demonstrated downstream harm. We use this finding to motivate a broader commentary on the potential impacts of bias in code generation, including the possibility that models may condition their outputs on identity information available from prior conversational context. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-22 in cs.SE). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.22511 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T07:08:15.250Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).