Metraipolis › archive › dc_70f8aebabc46

Not Your Diary, Just Your Topic: Grading the Wrong-User Control — arXiv:2611.09609

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.09609
License (as archived): CC0-1.0
Archived: 2026-11-09T22:34:13.045Z
SHA-256 of the archived text: 0d16dd73847a63782f709530c468544bcee4494e504578fb57a2b9826ab1bf15
Stable URL of this page: https://metraipolis.com/documento/dc_70f8aebabc46

How to cite

Not Your Diary, Just Your Topic: Grading the Wrong-User Control — arXiv:2611.09609 — arXiv (en), https://arxiv.org/abs/2611.09609 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_70f8aebabc46 (Metraipolis archive dc_70f8aebabc46, sha256 0d16dd73847a63782f709530c468544bcee4494e504578fb57a2b9826ab1bf15)

BibTeX

@misc{dc70f8aebabc46,
  title = {Not Your Diary, Just Your Topic: Grading the Wrong-User Control — arXiv:2611.09609},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09609},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_70f8aebabc46; sha256 0d16dd73847a63782f709530c468544bcee4494e504578fb57a2b9826ab1bf15},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09609. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A common test of personalization replaces a user's history with someone else's and measures the drop in accuracy. But this swap also changes the topics in the history, so the drop may reflect topic mismatch. We examine this problem with a frozen language model on LaMP-2 news categorization, using histories from similar users and synthetic histories assembled from other users' articles. The user's own history improves accuracy by 2.3 percentage points over a model trained without history. Removing history only at evaluation makes this gain appear larger, at 4.8 points. Category labels alone reproduce 86% of the accuracy gap between the user's own history and a random user's history. With synthetic histories matched on labels and text similarity, the gap falls to 0.9 points. Across two user splits and three training seeds, the advantage over a similar user's history is 3.2 points for users seen during training and 0.65 points for unseen users, where the evidence for an advantage remains inconclusive. A small movie-tagging experiment with one training seed also finds no advantage over other users' histories. These findings show why personalization should be evaluated on unseen users, with histories of similar content and a model trained without history as comparisons. We also find that adding gated cross-attention improves on a prefix-only model under our training setup; whether this holds more broadly remains open.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.09609 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:30:47.288Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).