Metraipolis › archive › dc_4f16a10cd550

On the Robustness of LLMs' Internal Representation of Code Correctness — arXiv:2608.08266

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08266
License (as archived)
CC0-1.0
Archived
2026-10-19T09:00:55.433Z
SHA-256 of the archived text
3a10fd05520d01276e56d4ce333a3dbb36998381415597eb6363b2fcfef28817
Stable URL of this page
https://metraipolis.com/documento/dc_4f16a10cd550

How to cite

On the Robustness of LLMs' Internal Representation of Code Correctness — arXiv:2608.08266 — arXiv (en), https://arxiv.org/abs/2608.08266 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4f16a10cd550 (Metraipolis archive dc_4f16a10cd550, sha256 3a10fd05520d01276e56d4ce333a3dbb36998381415597eb6363b2fcfef28817)
BibTeX
@misc{dc4f16a10cd550,
  title = {On the Robustness of LLMs' Internal Representation of Code Correctness — arXiv:2608.08266},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08266},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4f16a10cd550; sha256 3a10fd05520d01276e56d4ce333a3dbb36998381415597eb6363b2fcfef28817},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08266. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Code generated by modern language models often reads naturally. Yet, it also often fails to implement what was asked. This should be no surprise, as research shows the models' own confidence signals are poorly calibrated with actual correctness. A promising way to assess correctness looks inside the model: by contrasting the hidden states of correct and incorrect programs, recent work captured an internal signal of code correctness that is able to judge candidate solutions better than the model's token-level or stated confidence, with no test execution. However, this signal was captured under one particular way, leaving open an important question: whether it reflects a robust property of the model or an artifact of that choice. We study this question systematically, varying how the signal is extracted from the model internals. Besides this, we also ask if the signal's quality is limited by the data used to extract it, by constructing program pairs that differ only in the fault that makes them incorrect. Our results show that no single configuration is best, and that isolating the fault does not help.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-11 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.08266 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:00:31.480Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).