Metraipolis › archive › dc_3e4dabed92eb
Can Large Language Models Code Reality Criteria in Autobiographical Statements? Reliability and Validity of an Artificial Intelligence–Based Tool for Criteria-Based Content Analysis (2026) · DOI 10.3…
- Source
- Metraipolis archive
- Original
- https://doi.org/10.31234/osf.io/42ku6_v1
- License (as archived)
- CC-BY-4.0
- Archived
- 2026-10-18T06:29:18.292Z
- SHA-256 of the archived text
2a15a6c81c403064978b3613782cbacaa8f57091407d692c214322705fe3aa2c- Stable URL of this page
https://metraipolis.com/documento/dc_3e4dabed92eb
How to cite
Can Large Language Models Code Reality Criteria in Autobiographical Statements? Reliability and Validity of an Artificial Intelligence–Based Tool for Criteria-Based Content Analysis (2026) · DOI 10.3… — Metraipolis archive, https://doi.org/10.31234/osf.io/42ku6_v1 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_3e4dabed92eb (Metraipolis archive dc_3e4dabed92eb, sha256 2a15a6c81c403064978b3613782cbacaa8f57091407d692c214322705fe3aa2c)
BibTeX
@misc{dc3e4dabed92eb,
title = {Can Large Language Models Code Reality Criteria in Autobiographical Statements? Reliability and Validity of an Artificial Intelligence–Based Tool for Criteria-Based Content Analysis (2026) · DOI 10.3…},
howpublished = {Metraipolis archive},
url = {https://doi.org/10.31234/osf.io/42ku6_v1},
note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_3e4dabed92eb; sha256 2a15a6c81c403064978b3613782cbacaa8f57091407d692c214322705fe3aa2c},
year = {2026}
}Archived text
Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://doi.org/10.31234/osf.io/42ku6_v1. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Articolo recente con licenza aperta: scheda dai metadati Crossref (posted-content). Titolo: Can Large Language Models Code Reality Criteria in Autobiographical Statements? Reliability and Validity of an Artificial Intelligence–Based Tool for Criteria-Based Content Analysis Autori: Elsa Gewehr, Laureen Gräflich, Johannes Zimmermann, Renate Volbert Pubblicato: 2026-09-27 · editore: Center for Open Science DOI: https://doi.org/10.31234/osf.io/42ku6_v1 Licenza dell'articolo: CC-BY-4.0 Abstract (testo degli autori, CC-BY-4.0): Objective: Coding for Criteria-Based Content Analysis (CBCA) to differentiate fabricated from experience-based autobiographical statements is slow and depends on trained coders whose agreement is imperfect. We developed the large language model (LLM) based system CBC-AID and evaluate its reliability and validity in coding CBCA. Hypotheses: As preregistered, AI–human consistency is non-inferior to the interrater reliability of trained coders (H1: total score; H2: each criterion), and AI-based scores discriminate between experience-based and fabricated statements at least as well as human scores (H3: total score; H4: each criterion). Method: Two versions of an LLM-based coding tool (CBC-AID), a single-prompt and a multi-agent system, coded 957 transcripts from 399 persons in nine laboratory studies with human CBCA codings and known truth status. Bivariate Bayesian multilevel models estimated within-study correlations and standardized truth effects under global rating and weighted frequency. Results: AI–human correlations of the total scores (ρ = .53–.67) fell short of the benchmark of .66 in all four combinations of version and scoring scheme; Version 1 under global rating came closest (ρ = .67, 95% CrI [.62, .72]). The AI assigned higher scores than human coders (0.7–2.7 SD). Truth effects were similar for AI-based (δ = 0.17–0.32) and human coding (0.16–0.24; Δδ = −0.06 to 0.12); non-inferiority was supported for one of four combinations, and human coding itself discriminated weakly. Conclusions: LLM-based coding approximated but did not reach the consistency of trained coders and preserved their modest discriminative validity. It may serve as a traceable supplement to human coding in research; forensic use requires field validation and human oversight. Fonte: https://api.crossref.org/works/10.31234%2Fosf.io%2F42ku6_v1 — metadati Crossref, consultati il 2026-10-18. Interrogazione: «code generation language models».