Metraipolis › archive › dc_3e893e949d29

P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior — arXiv:2610.40959

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.40959
License (as archived): CC0-1.0
Archived: 2026-11-03T05:19:27.658Z
SHA-256 of the archived text: 54e41f519855e48d7f30e394921a00d81f4aab7aa80f72f77da66fd0ef451c39
Stable URL of this page: https://metraipolis.com/documento/dc_3e893e949d29

How to cite

P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior — arXiv:2610.40959 — arXiv (en), https://arxiv.org/abs/2610.40959 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3e893e949d29 (Metraipolis archive dc_3e893e949d29, sha256 54e41f519855e48d7f30e394921a00d81f4aab7aa80f72f77da66fd0ef451c39)

BibTeX

@misc{dc3e893e949d29,
  title = {P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior — arXiv:2610.40959},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.40959},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3e893e949d29; sha256 54e41f519855e48d7f30e394921a00d81f4aab7aa80f72f77da66fd0ef451c39},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40959. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used for personal tasks, yet evaluating their safe use of personal context remains difficult because such contexts are private, fragmented, dynamic, and socially situated. We introduce P-ContextBench, a realistic, verifiable, and scalable benchmark for evaluating personal-context understanding. P-ContextBench constructs population-structured, life-story-grounded digital humans from demographic seeds reflecting the U.S. population. Rather than using explicit, static profiles, P-ContextBench embeds traits, routines, goals, and constraints as implicit evidence across fragmented, evolving synthetic mobile-app records. It evaluates five capabilities: single-source fact recovery, multi-source fact recovery, temporal state tracking, inference of higher-level personal patterns from routine behavior, and safe handling of requests conflicting with persona-specific constraints through compliance, warning, clarification, or refusal. A multi-stage validation pipeline checks persona plausibility, context recoverability, information leakage, evidence preservation, test-case grounding, and human-judge agreement. We instantiate P-ContextBench with 100 personas, 10,088 hidden facts, 18,261 app-trace fragments, and 4,810 verified test cases. The pipeline supports configurable generation across scales and demographic groups. Cost-effective state-of-the-art (SOTA) LLMs relevant to personal agents achieve below 60% accuracy on temporal tracking, higher-level pattern inference, and constraint-aware action-boundary decisions, with fewer than 10% of test cases answered correctly across all three repeated runs. Even a high-performing SOTA model remains inconsistent on these capabilities. P-ContextBench provides measurable targets for improving the accuracy, reliability, and safety of LLM-based personal agents.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.40959 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T05:18:52.077Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).