Metraipolis › archive › dc_5280ec58e8be

Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization — arXiv:2608.11746

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11746
License (as archived)
CC0-1.0
Archived
2026-10-19T05:30:59.170Z
SHA-256 of the archived text
772566fcb17d1192158b7959a07dd1271a8359b32c83b3bc9b7141c487721771
Stable URL of this page
https://metraipolis.com/documento/dc_5280ec58e8be

How to cite

Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization — arXiv:2608.11746 — arXiv (en), https://arxiv.org/abs/2608.11746 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5280ec58e8be (Metraipolis archive dc_5280ec58e8be, sha256 772566fcb17d1192158b7959a07dd1271a8359b32c83b3bc9b7141c487721771)
BibTeX
@misc{dc5280ec58e8be,
  title = {Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization — arXiv:2608.11746},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11746},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5280ec58e8be; sha256 772566fcb17d1192158b7959a07dd1271a8359b32c83b3bc9b7141c487721771},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11746. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modern systems are increasingly expected to transfer across tasks not specified during training. What data facilitates generalization in these new, unanticipated settings? One hypothesis is that data with more structural information could contain shared circuits and subprograms that could be recycled in a wider array of downstream settings. Epiplexity, a recently proposed measure of the structural information a compute-bounded learner can extract from data, provides a mechanism to reason about this relationship. In this paper, we show how to operationalize epiplexity as an online training signal for data selection and synthetic data generation. For selection, we fit scaling laws to the training loss curves of natural data domains to predict the expected epiplexity gain as a function of training tokens, and use this signal to adaptively determine the sampling weights over domains during training. For synthetic data generation, we define a generator's reward as the change in learner epiplexity over a buffer of previously generated data and use REINFORCE policy gradients to guide the generator toward an epiplexity-maximizing distribution. In both cases, higher epiplexity predicts improved downstream performance on zero-shot and fine-tuning based tasks, supporting the hypothesis that data rich in structural information yield representations that transfer across domains.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.11746 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:15:39.250Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).