Metraipolis › archive › dc_91fb0af13aa2

Individual Text Corpora Predict User-Specific Knowledge: Benchmarks of Individualized Knowledge Simulation — arXiv:2609.08532

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.08532
License (as archived)
CC0-1.0
Archived
2026-10-16T15:05:25.742Z
SHA-256 of the archived text
4f554874abf4a9d1b3256d498475f071babe9cfecae6a2a94d69980badd2f1b7
Stable URL of this page
https://metraipolis.com/documento/dc_91fb0af13aa2

How to cite

Individual Text Corpora Predict User-Specific Knowledge: Benchmarks of Individualized Knowledge Simulation — arXiv:2609.08532 — arXiv (en), https://arxiv.org/abs/2609.08532 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_91fb0af13aa2 (Metraipolis archive dc_91fb0af13aa2, sha256 4f554874abf4a9d1b3256d498475f071babe9cfecae6a2a94d69980badd2f1b7)
BibTeX
@misc{dc91fb0af13aa2,
  title = {Individual Text Corpora Predict User-Specific Knowledge: Benchmarks of Individualized Knowledge Simulation — arXiv:2609.08532},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08532},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_91fb0af13aa2; sha256 4f554874abf4a9d1b3256d498475f071babe9cfecae6a2a94d69980badd2f1b7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08532. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This study examines whether individual text corpora (ICs) from search histories can be used to simulate individual knowledge. We collected ICs from 316 adults, who answered 36 multiple-choice knowledge items, and compared several large language models (LLMs) on this task, of which only Qwen3-1.7B proved viable. After task-specific fine-tuning via Low-Rank Adaptation (LoRA), Qwen3-1.7B outperformed both participants and a representative German norm sample on publicly available items. On non-public questions, however, the LLM performed worse than our participants, suggesting possible training data contamination for the public questions. When integrating ICs into retrieval-augmented generation to predict individual responses, LLM-participant Match accuracies significantly exceeded chance, which demonstrates a detectable individual knowledge signal. The probabilities assigned to the participants' answers were, however, low and far below the probability of correct answers, indicating poor calibration toward individual response patterns. Knowledge-gap prediction was sub-optimal, though it improved for corpora exceeding five million tokens. We discuss our entropy based evaluation benchmarks as calibration indices for individualized knowledge simulation.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-09 in cs.IR).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2609.08532 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T14:49:33.964Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).