Metraipolis › archive › dc_6418050eecc8

RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations — arXiv:2610.01780

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01780
License (as archived)
CC0-1.0
Archived
2026-10-06T12:06:05.722Z
SHA-256 of the archived text
1dc0594dcea4cf6b461be59cc20b3bea3224d9c3559bc8f37298ce5feb7939a8
Stable URL of this page
https://metraipolis.com/documento/dc_6418050eecc8

How to cite

RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations — arXiv:2610.01780 — arXiv (en), https://arxiv.org/abs/2610.01780 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6418050eecc8 (Metraipolis archive dc_6418050eecc8, sha256 1dc0594dcea4cf6b461be59cc20b3bea3224d9c3559bc8f37298ce5feb7939a8)
BibTeX
@misc{dc6418050eecc8,
  title = {RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations — arXiv:2610.01780},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01780},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6418050eecc8; sha256 1dc0594dcea4cf6b461be59cc20b3bea3224d9c3559bc8f37298ce5feb7939a8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01780. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An AI companion that talks with someone for months should come to understand them. It should know who they are, remember what they said, and recognize when something from the past matters now. Testing this needs real conversations, but real conversations are private, so existing benchmarks use invented people and invented questions. We release RealCompanion, ten real relationships between people and an AI companion, with 27,218 messages over up to 120 days. For each person, we release the full conversation, a profile, a persona, chat test items, and question test items. Every label points to the messages that support it, and every chat label comes with the reasoning that produced it. The real data shows three things. First, people rarely refer back. Only 3.4% of their messages depend on something said earlier, and when one does, the earlier message is usually far away (a median of 2,157 messages back). Averages hide this. Looking at the most recent messages finds the needed one 95.9% of the time overall, but only 2.2% of the time when it is far back. Second, AI systems cannot tell when the past matters. The detectors we tested barely beat chance on real messages, and when the same earlier messages are labeled "memories" instead of "earlier messages", models bring up the past 10 to 14 percentage points more often, even when nothing from the past is needed. Third, AI systems read more into a person than the person revealed. Three agent systems rebuild each persona equally well (F1 0.71). They see the person, and then imagine more. Understanding a person depends on knowing when their past matters and where what they shared ends, and only real conversations can test it.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.01780 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:27.974Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).