Metraipolis › archive › dc_7a3edf3ce88c

Parametric Multimodal User Memory: Storing What Captions Cannot Carry — arXiv:2608.28609

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.28609
License (as archived)
CC0-1.0
Archived
2026-10-17T07:47:04.710Z
SHA-256 of the archived text
1e79a2d30b2041986c4a87cfc4fbccf19e988d3259e134ae14f1c1de19491cf0
Stable URL of this page
https://metraipolis.com/documento/dc_7a3edf3ce88c

How to cite

Parametric Multimodal User Memory: Storing What Captions Cannot Carry — arXiv:2608.28609 — arXiv (en), https://arxiv.org/abs/2608.28609 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a3edf3ce88c (Metraipolis archive dc_7a3edf3ce88c, sha256 1e79a2d30b2041986c4a87cfc4fbccf19e988d3259e134ae14f1c1de19491cf0)
BibTeX
@misc{dc7a3edf3ce88c,
  title = {Parametric Multimodal User Memory: Storing What Captions Cannot Carry — arXiv:2608.28609},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.28609},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a3edf3ce88c; sha256 1e79a2d30b2041986c4a87cfc4fbccf19e988d3259e134ae14f1c1de19491cf0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28609. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A personalized agent needs a user memory: a persistent model of who its user is. Today it is almost always text -- transcripts and captions retrieved by similarity. This serves the captionable half of a person ("my cat is named Bibi"), but discards the perceptual half no caption can hold: how a voice sounds, how a face reads across age and lighting, how tired someone sounds. We measure this loss across five modalities: a strong caption-based re-identifier recovers as little as 0.11 of a dedicated encoder's recall, collapsing toward chance on non-nameable signals. We instead ground perceptual memory in the model, decomposing recall into two subproblems: a vision-language model grounds the referent in context (what and where), and a dedicated encoder extracts an identity key (who), stored as one inline token read by attention at generation with no external round-trip. Neither suffices alone -- the VLM identifies cross-age faces at only 0.54 recall where a face encoder reaches 0.81, and an ungrounded encoder recognizes a two-person-scene referent at 0.05 -- yet together they reach correct-region oracle (0.96), generalizing to multi-speaker audio and video. The recognition core is training-free: it reproduces the encoder's recall on any frozen model at O(1) registration cost. On PerceptMem (12 domains, 1,080 tasks) perceptual identity is capacity-limited while exact facts are binding-limited: identity belongs in a parametric bank, facts in a text store. The two memories compose cleanly: an agent with both can remember not only what its user said, but also what they are like.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.28609 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.654Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).