Metraipolis › archive › dc_7eca32e630d4

RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts — arXiv:2610.11761

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.11761
License (as archived)
CC0-1.0
Archived
2026-10-09T06:23:30.342Z
SHA-256 of the archived text
f91335678d3f0bfde3ebff4ba30e50da7173b94bc6d2e4531851fed396e4bcc8
Stable URL of this page
https://metraipolis.com/documento/dc_7eca32e630d4

How to cite

RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts — arXiv:2610.11761 — arXiv (en), https://arxiv.org/abs/2610.11761 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7eca32e630d4 (Metraipolis archive dc_7eca32e630d4, sha256 f91335678d3f0bfde3ebff4ba30e50da7173b94bc6d2e4531851fed396e4bcc8)
BibTeX
@misc{dc7eca32e630d4,
  title = {RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts — arXiv:2610.11761},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.11761},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7eca32e630d4; sha256 f91335678d3f0bfde3ebff4ba30e50da7173b94bc6d2e4531851fed396e4bcc8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11761. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The genome holds the blueprint that governs the biological properties of the cell. Consequently, advancing our knowledge of genomic function is crucial both for a broader understanding of biology and for continued biomedical advances. The success of large language models on natural language and protein sequences has motivated similar efforts on genomic data. However, standard genomic language models (gLMs) often require extremely large computational resources and still fall behind traditional methods on some downstream tasks. Recently, MSA-based pretraining has been proposed as an efficient alternative, but existing models are limited to short input contexts, restricting their use to short-range tasks, such as variant effect prediction. In this work, we present RAGenome, the first retrieval-based gLM that scales pretraining to longer contexts (100$\times$ longer than existing MSA-based gLMs), allowing it to capture both across-species evolutionary relationships and within-species longer-range interactions. Trained on whole-genome alignments from 100 vertebrates, RAGenome substantially improves the long-range capabilities of MSA-based gLMs, raising gene finding performance from 0.45 to 0.60, while remaining competitive on purely evolutionary-based tasks like prioritizing pathogenic variants. RAGenome provides competitive gLM performance at a fraction of the training cost, unifying evolutionary modeling and long-range capabilities within a single, flexible, scalable framework. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.11761 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:50:31.606Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).