Metraipolis › archive › dc_13206afe162c

UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces — arXiv:2611.02667

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02667
License (as archived)
CC0-1.0
Archived
2026-11-03T10:43:57.858Z
SHA-256 of the archived text
b04678ee2704fd32cd76460acbcb7757178a6de19f9c1e3e6c27a145f063e16b
Stable URL of this page
https://metraipolis.com/documento/dc_13206afe162c

How to cite

UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces — arXiv:2611.02667 — arXiv (en), https://arxiv.org/abs/2611.02667 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_13206afe162c (Metraipolis archive dc_13206afe162c, sha256 b04678ee2704fd32cd76460acbcb7757178a6de19f9c1e3e6c27a145f063e16b)
BibTeX
@misc{dc13206afe162c,
  title = {UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces — arXiv:2611.02667},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02667},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_13206afe162c; sha256 b04678ee2704fd32cd76460acbcb7757178a6de19f9c1e3e6c27a145f063e16b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02667. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Personalizing large language model assistants requires adapting their responses to individual users, including the selection of content, reasoning approach, level of detail, and form of presentation. Such preferences are often latent, becoming visible only when users respond to an initial answer and request revisions. We investigate whether these feedback trajectories can be converted into a reusable personalization signal without explicit preference profiles or labels, prompt-time access to interaction histories, or model parameter updates. We introduce UPSHIFT (User Preference Steering from Human-AI Interaction Feedback Traces), a training-free method that compares the hidden representations of initial and feedback-conditioned responses within each session, averages the resulting trajectory displacements across a user's sessions, and injects the aggregated preference direction when answering a new query. The method is motivated by the implicit-preference hypothesis: although an individual conversational transition may reflect topic progression or task-specific correction, changes that recur across sessions retain a component that captures how the user wants responses to be adjusted. We validate the preference-representation capability of UPSHIFT on PrefEval. In evaluations with 40 simulated users, every tested configuration improves preference alignment over the unsteered baseline on held-out topics for both Qwen3-8B and Qwen3-32B; the best configuration increases macro-averaged Elo from 1364.0 to 1605.2 and from 1341.9 to 1591.8, respectively. Matched-user and positive-versus-negative controls further demonstrate that these gains arise from the user-specific preference direction and its learned orientation. These results show that interaction histories can provide compact, reusable supervision for lightweight personalization.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02667 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:39:50.297Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).