Metraipolis › archive › dc_111f5102fe5e
Weak Ties, Strong Signals: Efficient Training Data Detection in Diffusion LLMs via Independent Token Sampling — arXiv:2609.22145
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22145
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:50:21.478Z
- SHA-256 of the archived text
3668899fb2e6a9bc40d4eeeb1d2b7cc5ef5c9e51b6255fc13b7c6e239f229f59- Stable URL of this page
https://metraipolis.com/documento/dc_111f5102fe5e
How to cite
Weak Ties, Strong Signals: Efficient Training Data Detection in Diffusion LLMs via Independent Token Sampling — arXiv:2609.22145 — arXiv (en), https://arxiv.org/abs/2609.22145 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_111f5102fe5e (Metraipolis archive dc_111f5102fe5e, sha256 3668899fb2e6a9bc40d4eeeb1d2b7cc5ef5c9e51b6255fc13b7c6e239f229f59)
BibTeX
@misc{dc111f5102fe5e,
title = {Weak Ties, Strong Signals: Efficient Training Data Detection in Diffusion LLMs via Independent Token Sampling — arXiv:2609.22145},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22145},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_111f5102fe5e; sha256 3668899fb2e6a9bc40d4eeeb1d2b7cc5ef5c9e51b6255fc13b7c6e239f229f59},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22145. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion large language models (dLLMs) offer a compelling alternative to autoregressive models, yet they may expose sensitive training data during denoising. Detecting such usage is challenging because dLLMs lack the efficient one-pass probability decomposition of causal architectures. Existing methods rely on random masking to obtain tractable token-wise detection signals under limited query budgets, but fail to control dependencies among masked tokens. We demonstrate that this token-wise approximation introduces a non-negative structural estimation error, which is theoretically characterized by the cumulative conditional mutual information (CMI) among masked tokens and can obscure subtle memorization signals. This insight suggests that reliable detection requires masked token sets with weak internal dependency. To avoid the prohibitive cost of directly estimating CMI over token combinations, we propose \textit{Independent Token Sampling} (ITS), a query-efficient framework that uses an attention-derived pairwise dependency proxy to approximate the CMI-aware selection criterion. ITS further incorporates a diversity-promoting strategy to improve token coverage across sampling rounds, yielding aggregated token-wise signals that are less affected by dependency-induced approximation error. Experiments on multiple datasets show that ITS consistently outperforms state-of-the-art baselines across different models and datasets, achieving an AUC improvement of 0.18 on the ArXiv dataset while maintaining strong performance under limited query budgets. The code is available at this https URL .
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.22145 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:45.085Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).