Metraipolis › archive › dc_a872850d60fa
LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs — arXiv:2609.01182
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.01182
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T06:41:31.376Z
- SHA-256 of the archived text
7930804ced6a5ac8e67f6e668165c0ee29f12d45344b7230afbb2655204a4eb2- Stable URL of this page
https://metraipolis.com/documento/dc_a872850d60fa
How to cite
LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs — arXiv:2609.01182 — arXiv (en), https://arxiv.org/abs/2609.01182 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a872850d60fa (Metraipolis archive dc_a872850d60fa, sha256 7930804ced6a5ac8e67f6e668165c0ee29f12d45344b7230afbb2655204a4eb2)
BibTeX
@misc{dca872850d60fa,
title = {LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs — arXiv:2609.01182},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.01182},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a872850d60fa; sha256 7930804ced6a5ac8e67f6e668165c0ee29f12d45344b7230afbb2655204a4eb2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01182. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Flagship language models appear saturated on benchmarks like MMLU (Hendrycks et al., 2021), scoring above 90% - yet benchmarks test only what the experimenter thought to ask, the availability bias of fixed question sets. LLMPEDIA makes this bias measurable and browsable. We recursively materialized ~1.3M articles from three model families' parametric memory (GPT-5-mini, DeepSeek-V3.2, Llama-3.3-70B) without retrieval, then audited a stratified sample of atomic claims against Wikipedia and a curated web stack, coloring every claim supported, refuted, or insufficient (Saeed and Razniewski, 2026). On a uniform random sample the true rate is 68.4% - more than 21 pp below MMLU - with 30.5% of claims insufficient: assertions no benchmark probes and the world's largest encyclopedia cannot adjudicate - long-tail knowledge or plausible hallucination, the evidence cannot tell - extending to free text the coverage gap GPTKB established for triples (Hu et al., 2025). The resulting live, open encyclopedia lets visitors inspect this frontier one claim at a time through five one-click views - link-traversal exploration, claim-level factuality, cross-model and political-persona comparison, and a guided topic drill-down - each page, claim, and verdict at a stable URL. LLMPEDIA is live at this https URL Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.01182 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T06:33:38.848Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).