Metraipolis › archive › dc_d6516cc9b4e2

UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces — arXiv:2611.02667

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02667
License (as archived)
CC0-1.0
Archived
2026-11-03T10:53:39.662Z
SHA-256 of the archived text
b02bf6303108ec94f1165e262cff1a15cd91984302b23f4c38b2846ba69611c9
Stable URL of this page
https://metraipolis.com/documento/dc_d6516cc9b4e2

How to cite

UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces — arXiv:2611.02667 — arXiv (en), https://arxiv.org/abs/2611.02667 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d6516cc9b4e2 (Metraipolis archive dc_d6516cc9b4e2, sha256 b02bf6303108ec94f1165e262cff1a15cd91984302b23f4c38b2846ba69611c9)
BibTeX
@misc{dcd6516cc9b4e2,
  title = {UPSHIFT: User Preference Steering from Human-AI Interaction Feedback Traces — arXiv:2611.02667},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02667},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d6516cc9b4e2; sha256 b02bf6303108ec94f1165e262cff1a15cd91984302b23f4c38b2846ba69611c9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02667. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Personalizing large language model assistants requires adapting their responses to individual users, including the selection of content, reasoning approach, level of detail, and form of presentation. Such preferences are often latent, becoming visible only when users respond to an initial answer and request revisions. We investigate whether these feedback trajectories can be converted into a reusable personalization signal without explicit preference profiles or labels, prompt-time access to interaction histories, or model parameter updates. We introduce UPSHIFT (User Preference Steering from Human-AI Interaction Feedback Traces), a training-free method that compares the hidden representations of initial and feedback-conditioned responses within each session, averages the resulting trajectory displacements across a user's sessions, and injects the aggregated preference direction when answering a new query. The method is motivated by the implicit-preference hypothesis: although an individual conversational transition may reflect topic progression or task-specific correction, changes that recur across sessions retain a component that captures how the user wants responses to be adjusted. We validate the preference-representation capability of UPSHIFT on PrefEval. In evaluations with 40 simulated users, every tested configuration improves preference alignment over the unsteered baseline on held-out topics for both Qwen3-8B and Qwen3-32B; the best configuration increases macro-averaged Elo from 1364.0 to 1605.2 and from 1341.9 to 1591.8, respectively. Matched-user and positive-versus-negative controls further demonstrate that these gains arise from the user-specific preference direction and its learned orientation. These results show that interaction histories can provide compact, reusable supervision for lightweight personalization.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02667 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.373Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).