Metraipolis › archive › dc_e99bed54e6b5

Representation Risk in Pretrained Image Encoders — arXiv:2609.35470

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35470
License (as archived)
CC0-1.0
Archived
2026-10-07T14:46:44.871Z
SHA-256 of the archived text
bf684a99ef1d235422e6f9be5407d8648a5db5d198703c94f862adb9a2e74e19
Stable URL of this page
https://metraipolis.com/documento/dc_e99bed54e6b5

How to cite

Representation Risk in Pretrained Image Encoders — arXiv:2609.35470 — arXiv (en), https://arxiv.org/abs/2609.35470 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e99bed54e6b5 (Metraipolis archive dc_e99bed54e6b5, sha256 bf684a99ef1d235422e6f9be5407d8648a5db5d198703c94f862adb9a2e74e19)
BibTeX
@misc{dce99bed54e6b5,
  title = {Representation Risk in Pretrained Image Encoders — arXiv:2609.35470},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35470},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e99bed54e6b5; sha256 bf684a99ef1d235422e6f9be5407d8648a5db5d198703c94f862adb9a2e74e19},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35470. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Applied researchers increasingly convert images into features with pretrained encoders, then use those features in a downstream prediction model. The encoder is often treated as an implementation detail. We show that it can instead be a consequential source of model uncertainty. We call this uncertainty representation risk: plausible pretrained encoders map the same images into different feature spaces and can yield sharply different out-of-sample conclusions from predictive performance. We compare ten modern and legacy frozen encoders across applications involving house prices, racehorse performance, breast-cancer histology, chest radiographs, continuous facial age, and rice disease. With common dimension control, heads, and group-safe splits, validation selects SigLIP 2 for houses, raising test $R^2$ from 0.396 for ResNet50 to 0.629, and DINOv2 for horses, raising $R^2$ from 0.029 to 0.105. No encoder is best in every task. Candidate procedures are constructed using training data and compared on a separate validation partition. The selected procedure reaches 0.658 for houses and 0.979 accuracy for pneumonia. Fixed-split gains are small for horses and rice, while repeated partitions reveal instability in horse feature union. Continuous age selects SigLIP 2 at 4.786 years MAE. The principal representation gaps persist with neural heads, similarly sized DINOv2 and ViT models, and limited adaptation. These results support a simple workflow: benchmark plausible representations, select on locked validation data, combine only when separate validation evidence justifies the additional cost, and report paired and split-level uncertainty. We implement this workflow in LOOKAGAIN-ML, the software package used to conduct the analyses in this paper.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CV).
Categorie arXiv: General Economics (econ.GN) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.35470 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T14:19:00.880Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).