Metraipolis › archive › dc_c61869c12e98
Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study — arXiv:2609.32638
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32638
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:55:48.224Z
- SHA-256 of the archived text
6bf6ff93c9abaed65f7a3c533b01b8dbbba5b7e39b5d129f584768e6ea0aab45- Stable URL of this page
https://metraipolis.com/documento/dc_c61869c12e98
How to cite
Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study — arXiv:2609.32638 — arXiv (en), https://arxiv.org/abs/2609.32638 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c61869c12e98 (Metraipolis archive dc_c61869c12e98, sha256 6bf6ff93c9abaed65f7a3c533b01b8dbbba5b7e39b5d129f584768e6ea0aab45)
BibTeX
@misc{dcc61869c12e98,
title = {Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study — arXiv:2609.32638},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32638},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c61869c12e98; sha256 6bf6ff93c9abaed65f7a3c533b01b8dbbba5b7e39b5d129f584768e6ea0aab45},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32638. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) are increasingly used to generate synthetic survey respondents and digital twins of real people, but whether their output preserves real human statistical structure, rather than surface plausibility, remains unresolved, and most existing evidence comes from proprietary models rather than open-weight ones. We evaluate three open-weight LLM families on a cross-instrument calibration task: conditioning personas on real respondents' verbatim answers to one psychometric instrument and measuring them on a second, construct-distance-controlled instrument, checked against a 2,058-person human panel. Across a 139-pair grid, the simulated cross-instrument correlation tracks the real human correlation at r = 0.70 - 0.73 in every model, driven mainly by correct sign rather than precise magnitude and concentrated in pairs of moderate construct distance. A correlation of this magnitude, obtained from untuned open-weight models conditioned only on individual-level survey data, is a substantively encouraging result for LLM-based behavioral simulation and digital-twin applications: specific model families and releases already reproduce a meaningful share of real human cross-instrument structure without any fine-tuning. This capability does not, however, improve monotonically across model releases: on a matched panel, the newest of three tested Llama releases performs worst on two of three headline metrics, so realizing its promise in practice requires release-specific, distance-aware verification rather than a one-time benchmark. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.32638 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:00.892Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).