Metraipolis › archive › dc_6efdaea3e3fe

SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data — arXiv:2609.12353

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12353
License (as archived)
CC0-1.0
Archived
2026-10-16T03:08:19.682Z
SHA-256 of the archived text
6ed7da8a869374cfc869f4856188d793cdff9ed830374bb58b0619f399f6755d
Stable URL of this page
https://metraipolis.com/documento/dc_6efdaea3e3fe

How to cite

SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data — arXiv:2609.12353 — arXiv (en), https://arxiv.org/abs/2609.12353 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6efdaea3e3fe (Metraipolis archive dc_6efdaea3e3fe, sha256 6ed7da8a869374cfc869f4856188d793cdff9ed830374bb58b0619f399f6755d)
BibTeX
@misc{dc6efdaea3e3fe,
  title = {SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data — arXiv:2609.12353},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12353},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6efdaea3e3fe; sha256 6ed7da8a869374cfc869f4856188d793cdff9ed830374bb58b0619f399f6755d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12353. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models trained recursively on their own or other models' outputs undergo model collapse, in which distributional tails and factual accuracy deteriorate while fluency survives. Prior work diagnoses collapse after training; the actionable problem is screening a corpus of unknown provenance before training. We introduce SynthSentry, a corpus-level, model-agnostic contamination signal requiring no access to the generating model, no generation history, and no synthetic labels. The score is a distributional divergence over three statistics: lexical diversity collapse, n-gram tail truncation, and perplexity variance across reference models. We evaluate on corpora contaminated by small open-weight generators and an instruction-tuned open-weight model under a leave-one-generator-out protocol. A domain-stratified study measures false positives on naturally repetitive human text (legal, clinical, source code). The score ranks corpora by severity with little loss when whole generator families are held out. Per-domain calibration holds near its nominal false-positive budget once covariance shrinkage and a bootstrap threshold replace a naive quantile, which runs four times over budget. A downstream fine-tuning check showed no contamination-driven accuracy deficit at our scale, so whether pruning recovers one remains open; the same run shows over-pruning risk once pruning exceeds the true contamination fraction. We frame screening as a data-curation defense rather than a post-hoc diagnosis and release the scoring toolkit. All results are small-scale; scope is English-language, batch-mode corpus screening. Contamination sources are single-generation or hand-authored rather than recursively generated, so results speak to synthetic contamination generally and not to recursion depth.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.12353 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:02:47.536Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).