Metraipolis › archive › dc_69283fc92da0
Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation — arXiv:2608.26159
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.26159
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T14:24:11.853Z
- SHA-256 of the archived text
dada428ad42ee80d0bcbe094b3adf389e79d53a4ce89982550312d7b1e2c4666- Stable URL of this page
https://metraipolis.com/documento/dc_69283fc92da0
How to cite
Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation — arXiv:2608.26159 — arXiv (en), https://arxiv.org/abs/2608.26159 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_69283fc92da0 (Metraipolis archive dc_69283fc92da0, sha256 dada428ad42ee80d0bcbe094b3adf389e79d53a4ce89982550312d7b1e2c4666)
BibTeX
@misc{dc69283fc92da0,
title = {Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation — arXiv:2608.26159},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.26159},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_69283fc92da0; sha256 dada428ad42ee80d0bcbe094b3adf389e79d53a4ce89982550312d7b1e2c4666},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26159. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Self-Generated Text Recognition (SGTR)--the ability of an LLM to identify its own outputs--poses risks to AI safeguards that rely on LLMs as evaluators or monitors: an LLM may recognize outputs from other copies of the same model and make biased judgments or collude outright. Prior work has drawn conflicting conclusions about whether current models possess significant SGTR capabilities. We explain these disagreements by identifying key experimental design choices--which we term operationalizations--that drive divergent results. Evaluating 13-21 models across six presentation operationalizations and four task-domain operationalizations, we find that accuracy varies substantially with evaluation format (pairwise vs individual assessments of text), conversation format (presenting candidate text in user tags vs assistant tags), and the domain of the task used to generate candidate text (e.g., coding vs summarization). We corroborate previous observations that a quality heuristic--models attributing authorship to text they perceive as higher quality--is a dominant confound. We also find that improving a model's SGTR performance via supervised fine-tuning (SFT) on one operationalization can generalize to others, and can increase the model's preference for its own outputs when it acts as a judge in the AlpacaEval framework. Our results suggest that, despite confounds, some models possess practical SGTR capabilities, and that SGTR should be monitored and considered in the design of safety-critical AI applications. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.26159 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T13:16:53.596Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).