Metraipolis › archive › dc_7abde5ba5e26

Program Bindings Inside Code Models — arXiv:2610.39489

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39489
License (as archived)
CC0-1.0
Archived
2026-11-01T02:29:03.530Z
SHA-256 of the archived text
3d28b7b3bfe8e77c77747bf33e8d410ab50d03739dbccf2bd7722b3e384da6dc
Stable URL of this page
https://metraipolis.com/documento/dc_7abde5ba5e26

How to cite

Program Bindings Inside Code Models — arXiv:2610.39489 — arXiv (en), https://arxiv.org/abs/2610.39489 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7abde5ba5e26 (Metraipolis archive dc_7abde5ba5e26, sha256 3d28b7b3bfe8e77c77747bf33e8d410ab50d03739dbccf2bd7722b3e384da6dc)
BibTeX
@misc{dc7abde5ba5e26,
  title = {Program Bindings Inside Code Models — arXiv:2610.39489},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39489},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7abde5ba5e26; sha256 3d28b7b3bfe8e77c77747bf33e8d410ab50d03739dbccf2bd7722b3e384da6dc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39489. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Code language models complete, review, and repair programs without exposing the program structure encoded in their hidden states. In this paper, we test whether the hidden states of a code model can be described and edited using the information a compiler front end computes. For each token of a Python program, standard analyzers provide its position in the abstract syntax tree, its scope, its definition-use links, and its syntactic category. We fit a decomposition that expresses every hidden state of a frozen model as a sum of one term per token, each combining the token with this structural label, and we replace the model's own states with the decomposition at every layer while it predicts the output of a small program. If the model still answers correctly, the decomposition preserves the information the model uses. We apply this test to 14 open models, seven of them trained for code, and compare against a baseline decomposition of the same size built from token positions. The static-analysis decomposition preserves the model's answers better than the positional one in 10 of the 14 models, including six of the seven code models, and deleting the definition-use links from it removes most of that advantage on the two models tested. Editing the decomposition, for instance rebinding the argument of a call, makes StarCoder2-15B produce the answer of the edited program in 90 of 100 cases, compared with 19 for the positional baseline. The Qwen2.5 family is the exception on several measures. For most of the models tested, standard static analysis therefore provides both a description of what the hidden states encode about a program and a way to change it without retraining.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.39489 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-01T02:12:58.492Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).