Metraipolis › archive › dc_b430e8ff06eb

ALICE: In-context, Zero-shot, Mutual Information Estimation — arXiv:2609.34962

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34962
License (as archived)
CC0-1.0
Archived
2026-10-07T22:38:41.816Z
SHA-256 of the archived text
2a714543dff9f64d4a0fb0f2f3e5068ea2b7e430fab56745f6cc36bbdaeb87c2
Stable URL of this page
https://metraipolis.com/documento/dc_b430e8ff06eb

How to cite

ALICE: In-context, Zero-shot, Mutual Information Estimation — arXiv:2609.34962 — arXiv (en), https://arxiv.org/abs/2609.34962 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b430e8ff06eb (Metraipolis archive dc_b430e8ff06eb, sha256 2a714543dff9f64d4a0fb0f2f3e5068ea2b7e430fab56745f6cc36bbdaeb87c2)
BibTeX
@misc{dcb430e8ff06eb,
  title = {ALICE: In-context, Zero-shot, Mutual Information Estimation — arXiv:2609.34962},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34962},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b430e8ff06eb; sha256 2a714543dff9f64d4a0fb0f2f3e5068ea2b7e430fab56745f6cc36bbdaeb87c2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34962. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Estimating mutual information (MI) from samples is a central objective in a variety of scientific fields. Modern neural estimators are accurate in the large-data regime, but they fall short when data is scarce, and each must be fit anew for every distribution under study. Current estimators are moreover tied to specific data types. These constraints limit their adoption in many applications where per-distribution training is impractical and sample sizes are small. We present ALICE, a foundation model that removes per-distribution training, while achieving competitive estimation accuracy. Trained exclusively on a broad family of synthetic distributions, ALICE acts as an in-context estimator of rectified-flow velocity fields: conditioned on samples of an unseen distribution, it estimates that distribution's velocity field without any explicit training. MI is then obtained through a fixed identity that integrates the squared difference between the joint and conditional fields. We validate ALICE on a standard, challenging benchmark and apply it in three domains, biology, genetics, and neuroscience, whose data the model has never seen. For the first time, we show that a single model closes the gap with neural estimators trained separately for each distribution, while natively supporting different data dimensionality and sample cardinality, enabling zero-shot MI analysis across scientific domains.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.34962 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:28.315Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).