Metraipolis › archive › dc_3d1c0fb021bd
Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian — arXiv:2611.05385
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.05385
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:18:34.932Z
- SHA-256 of the archived text
9173d82b37218a1aeb159cec4c65885853091e457f79f5cf814ea591ad0bd716- Stable URL of this page
https://metraipolis.com/documento/dc_3d1c0fb021bd
How to cite
Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian — arXiv:2611.05385 — arXiv (en), https://arxiv.org/abs/2611.05385 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3d1c0fb021bd (Metraipolis archive dc_3d1c0fb021bd, sha256 9173d82b37218a1aeb159cec4c65885853091e457f79f5cf814ea591ad0bd716)
BibTeX
@misc{dc3d1c0fb021bd,
title = {Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian — arXiv:2611.05385},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.05385},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3d1c0fb021bd; sha256 9173d82b37218a1aeb159cec4c65885853091e457f79f5cf814ea591ad0bd716},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05385. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Automated detection of machine-generated text is applied where the verdict carries a cost for the author, yet origin is a property of the production process, not an observable property of the text. The conditions under which a text is produced push it in the same direction in which a generative model is presumed to push it. This study measures the contribution of these conditions while origin is held fixed. The Russian-language corpus of 1882 documents comprises a machine part, generated by four generation channels (each backed by its own model family) across 45 tasks in three task modes, and an archival human part annotated for source regimentation level before the texts were read. The primary contrast is computed within the machine corpus: both halves of a pair are generated by the same model for the same task and differ only in task wording. Machine-likeness is measured by four operationalisations: an interpretable index with weights fixed before data collection, a trained classifier, a zero-shot metric on an open model, a judge model. The main result is disagreement between procedures: changing the task mode shifted the scores of all four, but three gave a negative shift, the judge a positive one, and none of the four intervals covers zero. The comparison plan and the specifications of procedures 2-4 were fixed after the result of procedure 1, before procedures 2-4 ran; the disagreement survived correction of a corpus defect and of the nested validation. A baseline on the source identifier alone reaches a median AUROC of 1.0000 on the thirteen group splits where it is defined, and removing emphasis markers flipped a confident machine verdict to human without changing the full text by a single character. The scores are properties of a specific measurement system and corpus: they reflect data collection, repetition, formatting, and preprocessing decisions as well as linguistic style. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.05385 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:13:03.936Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).