Metraipolis › archive › dc_f5a5285d22cd

A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction — arXiv:2611.05849

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05849
License (as archived)
CC0-1.0
Archived
2026-11-05T18:22:03.932Z
SHA-256 of the archived text
dc17266c976945894dc9d7b51756eced033cc98f3dd51890ec198f6ccbe83c2f
Stable URL of this page
https://metraipolis.com/documento/dc_f5a5285d22cd

How to cite

A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction — arXiv:2611.05849 — arXiv (en), https://arxiv.org/abs/2611.05849 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f5a5285d22cd (Metraipolis archive dc_f5a5285d22cd, sha256 dc17266c976945894dc9d7b51756eced033cc98f3dd51890ec198f6ccbe83c2f)
BibTeX
@misc{dcf5a5285d22cd,
  title = {A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction — arXiv:2611.05849},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05849},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f5a5285d22cd; sha256 dc17266c976945894dc9d7b51756eced033cc98f3dd51890ec198f6ccbe83c2f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05849. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Foundation-model predictors of perturbation response are evaluated primarily by aggregate accuracy. Using their predictions to prioritize experiments requires assessing each prediction's reliability. We audit the state-transition (ST) decoder used in STATE across three encoders (SE-600M, scGPT, and Tahoe-x1) and four cell lines under CRISPRi. The decoder's first attention layer has a low-rank output. Projecting this output onto 4-32 fixed directions estimated from training activations preserves mean per-perturbation Pearson delta correlation (PDC). This low-rank attention bottleneck bounds the decoder's reliability by distance to training responses: PDC declines as the cosine distance from a perturbation's observed response to its nearest training response increases. We propose predicted-NN-distance, the cosine distance from a predicted response to its nearest training response, as a per-prediction confidence score. The score has a mean absolute Spearman correlation of 0.81 with PDC, close to the 0.88 obtained with the observed response. Thresholds calibrated on this score transfer across encoders and cell lines, and the score remains predictive of PDC for drug perturbations.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2611.05849 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:15:49.006Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).