Metraipolis › archive › dc_0b6287ccee69

P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior — arXiv:2610.40959

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.40959
License (as archived): CC0-1.0
Archived: 2026-11-02T07:17:27.010Z
SHA-256 of the archived text: 6a5942240beff2854b66465d1c46a2df99c07b26a0168a1dc965c4d175cb958e
Stable URL of this page: https://metraipolis.com/documento/dc_0b6287ccee69

How to cite

P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior — arXiv:2610.40959 — arXiv (en), https://arxiv.org/abs/2610.40959 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0b6287ccee69 (Metraipolis archive dc_0b6287ccee69, sha256 6a5942240beff2854b66465d1c46a2df99c07b26a0168a1dc965c4d175cb958e)

BibTeX

@misc{dc0b6287ccee69,
  title = {P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior — arXiv:2610.40959},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.40959},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0b6287ccee69; sha256 6a5942240beff2854b66465d1c46a2df99c07b26a0168a1dc965c4d175cb958e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40959. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used for personal tasks, yet evaluating their safe use of personal context remains difficult because such contexts are private, fragmented, dynamic, and socially situated. We introduce P-ContextBench, a realistic, verifiable, and scalable benchmark for evaluating personal-context understanding. P-ContextBench constructs population-structured, life-story-grounded digital humans from demographic seeds reflecting the U.S. population. Rather than using explicit, static profiles, P-ContextBench embeds traits, routines, goals, and constraints as implicit evidence across fragmented, evolving synthetic mobile-app records. It evaluates five capabilities: single-source fact recovery, multi-source fact recovery, temporal state tracking, inference of higher-level personal patterns from routine behavior, and safe handling of requests conflicting with persona-specific constraints through compliance, warning, clarification, or refusal. A multi-stage validation pipeline checks persona plausibility, context recoverability, information leakage, evidence preservation, test-case grounding, and human-judge agreement. We instantiate P-ContextBench with 100 personas, 10,088 hidden facts, 18,261 app-trace fragments, and 4,810 verified test cases. The pipeline supports configurable generation across scales and demographic groups. Cost-effective state-of-the-art (SOTA) LLMs relevant to personal agents achieve below 60% accuracy on temporal tracking, higher-level pattern inference, and constraint-aware action-boundary decisions, with fewer than 10% of test cases answered correctly across all three repeated runs. Even a high-performing SOTA model remains inconsistent on these capabilities. P-ContextBench provides measurable targets for improving the accuracy, reliability, and safety of LLM-based personal agents.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.40959 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:16:57.130Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).