Metraipolis › archive › dc_d45fee47605a

Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates — arXiv:2608.29455

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29455
License (as archived)
CC0-1.0
Archived
2026-10-17T08:06:59.681Z
SHA-256 of the archived text
a177cde4ce8ca3d8382851e990c73b06180e74f7b2b91443fec23be81437890c
Stable URL of this page
https://metraipolis.com/documento/dc_d45fee47605a

How to cite

Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates — arXiv:2608.29455 — arXiv (en), https://arxiv.org/abs/2608.29455 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d45fee47605a (Metraipolis archive dc_d45fee47605a, sha256 a177cde4ce8ca3d8382851e990c73b06180e74f7b2b91443fec23be81437890c)
BibTeX
@misc{dcd45fee47605a,
  title = {Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates — arXiv:2608.29455},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29455},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d45fee47605a; sha256 a177cde4ce8ca3d8382851e990c73b06180e74f7b2b91443fec23be81437890c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29455. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLMs are increasingly used as human surrogates, often on the premise that richer persona data could make them substitutes or exploratory tools for specific individuals. We test this premise across four datasets covering more than 400,000 participants and more than 6,000 survey items and experimental outcomes. LLMs perform well at the aggregate level: their average responses closely align with average human responses to the same items. But this success largely reflects predicting each item's average human response. Once each item's human mean is removed, LLM predictions explain only 3.05% of the remaining respondent-specific variation, far below the 53.6% human test-retest benchmark. Richer personas, model variants, and fine-tuning do not close this gap. In variance analyses, once item means are removed, the reliable remaining signal is person-by-item. It captures how a respondent departs from the mean on a particular item and is about 8.9x larger than the stable person effect. Persona data encode the respondent, but not this item-specific deviation. LLM responses also compress human response distributions, using less spread, fewer response categories, and distorted distributional shapes. We call this pattern item-mean surrogacy. Current LLM surrogates can approximate item averages, but not the distributions or respondent-specific deviations needed to replace individual humans. We propose four empirical tests for LLM-based human-surrogate claims.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Computers and Society (cs.CY); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.29455 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:47:52.700Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).