Metraipolis › archive › dc_d1bef9024972
HistoTx: Early fusion of H&E images and spatial transcriptomics at varying spatial transcriptomics resolution with self-supervised learning — arXiv:2611.02681
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02681
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:58:35.611Z
- SHA-256 of the archived text
9968955af57f3692ce7b381fd99b8d4712deaec86892ed973e59d8d76549a318- Stable URL of this page
https://metraipolis.com/documento/dc_d1bef9024972
How to cite
HistoTx: Early fusion of H&E images and spatial transcriptomics at varying spatial transcriptomics resolution with self-supervised learning — arXiv:2611.02681 — arXiv (en), https://arxiv.org/abs/2611.02681 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d1bef9024972 (Metraipolis archive dc_d1bef9024972, sha256 9968955af57f3692ce7b381fd99b8d4712deaec86892ed973e59d8d76549a318)
BibTeX
@misc{dcd1bef9024972,
title = {HistoTx: Early fusion of H&E images and spatial transcriptomics at varying spatial transcriptomics resolution with self-supervised learning — arXiv:2611.02681},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02681},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d1bef9024972; sha256 9968955af57f3692ce7b381fd99b8d4712deaec86892ed973e59d8d76549a318},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02681. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Spatial transcriptomics maps gene expression across tissue sections and, when integrated with whole slide images, jointly captures both morphological architecture and molecular activity to provide a rich, multimodal view of tissue biology. ML approaches combining spatial transcriptomics and WSIs typically learn a mapping from morphological features to molecular profiles via contrastive alignment or supervised fine-tuning of pathology foundation models. In both cases, transcriptomics serves as a training signal rather than a true input modality, leaving inference fundamentally image-driven. We propose an early-fusion vision transformer architecture in which transcript tokens are constructed at arbitrary spatial resolution, merged with image patch tokens, and jointly processed through a shared transformer stack for deep cross-modal interaction. Starting from a pretrained vision-only pathology foundation model, we train HistoTx by continuing self-supervised pretraining on paired image and spatial transcriptomics data and demonstrate that: 1) at inference with image only, performance remains on par with the baseline on most tasks while improving on tasks that inherently benefit from molecular knowledge, such as gene expression prediction; 2) when both modalities are available at inference, jointly providing image and transcript inputs outperforms either modality alone. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Genomics (q-bio.GN); Quantitative Methods (q-bio.QM). Pagina del lavoro: https://arxiv.org/abs/2611.02681 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:49:38.667Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).