Metraipolis › archive › dc_40ff73ce58ac

Generative Latent Priors for Interpreting and Steering Protein Language Models — arXiv:2611.09818

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09818
License (as archived)
CC0-1.0
Archived
2026-11-09T22:34:17.916Z
SHA-256 of the archived text
2ace46a2e82b159af425df46dcf91808ddb36a88a99d8aa7226ff3718e5ecfb8
Stable URL of this page
https://metraipolis.com/documento/dc_40ff73ce58ac

How to cite

Generative Latent Priors for Interpreting and Steering Protein Language Models — arXiv:2611.09818 — arXiv (en), https://arxiv.org/abs/2611.09818 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40ff73ce58ac (Metraipolis archive dc_40ff73ce58ac, sha256 2ace46a2e82b159af425df46dcf91808ddb36a88a99d8aa7226ff3718e5ecfb8)
BibTeX
@misc{dc40ff73ce58ac,
  title = {Generative Latent Priors for Interpreting and Steering Protein Language Models — arXiv:2611.09818},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09818},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40ff73ce58ac; sha256 2ace46a2e82b159af425df46dcf91808ddb36a88a99d8aa7226ff3718e5ecfb8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09818. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dominant methods for interpreting large models impose strong structural assumptions. Generative Latent Priors (GLP) are a complementary approach: a diffusion model that learns the distribution of a frozen model's activations directly, imposing no assumptions. We port GLP to ESM-2 activations (UniRef50) and evaluate it at two host scales (ESM-2-8M and -650M), both as a lens for protein representations and as a prior for editing them. Generation quality scales smoothly with depth and compute. As a lens, single GLP "meta-neurons" are the strongest probes of discrete Swiss-Prot concepts at both hosts, beating InterPLM's released SAE features, and lead on continuous properties at 8M though not at 650M; falsification tests show this is mostly concentration, not new information. As an editing prior, the on-manifold correction makes DiffMean steering of three biochemical properties effective and restores on-manifoldness, but only in isolation: re-embedding the sequences actually produced after several rounds of decoding, the structural plausibility is compromised. This is mainly reproduced in additional SAE-based clamping structural steering experiments, where only Helix motif succeeds. The naturalness failure is traced to the rewriting coverage and the decoder rather than the prior, highlighting what the method must clear before it can serve for both discovery-by-editing and discovery-by-exploration. Trained GLPs are released on this http URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.09818 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:30:47.293Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).