Metraipolis › archive › dc_0dfc72ec6022
RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces — arXiv:2610.05852
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05852
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:31:17.182Z
- SHA-256 of the archived text
d9b8a86f4c5ba26ce9acc94e416ff4f8750959638d7a00f9a230bd70df7d4743- Stable URL of this page
https://metraipolis.com/documento/dc_0dfc72ec6022
How to cite
RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces — arXiv:2610.05852 — arXiv, https://arxiv.org/abs/2610.05852 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0dfc72ec6022 (Metraipolis archive dc_0dfc72ec6022, sha256 d9b8a86f4c5ba26ce9acc94e416ff4f8750959638d7a00f9a230bd70df7d4743)
BibTeX
@misc{dc0dfc72ec6022,
title = {RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces — arXiv:2610.05852},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05852},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0dfc72ec6022; sha256 d9b8a86f4c5ba26ce9acc94e416ff4f8750959638d7a00f9a230bd70df7d4743},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05852. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Frozen representations are widely reused for downstream classification, yet each new task typically requires fitting a new predictor. We ask whether the few-shot prediction procedure itself can instead be learned once and reused across datasets and representation spaces. To study this question, we introduce RepShiftBench, comprising 1,218 encoder--dataset tasks across text, image, and audio, with separate evaluation of generalization to unseen datasets, unseen encoders, jointly unseen datasets and encoders, and unseen modalities. The benchmark exposes a substantial gap: Logistic Regression fitted independently on each episode outperforms every evaluated in-context learner across all settings. We introduce RepICL, a meta-trained in-context learner that canonicalizes each episode through episodic whitening before prediction. Its inductive variant, RepICL-I, surpasses Logistic Regression in all 12 benchmark settings, while RepICL-T substantially outperforms existing transductive methods. Ablations identify episodic whitening as the primary source of these gains, while showing that it is not a universally beneficial preprocessing step. Across both variants, the gains concentrate on queries for which simple support prototypes favor the wrong class or provide little separation between the true class and competing classes. Transduction provides its largest additional gains when limited support coverage gives a misleading view of class separation. Together, these results demonstrate that a shared few-shot prediction procedure can generalize beyond the representation spaces observed during training. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.05852 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:27:05.829Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).