Metraipolis › archive › dc_66c1745524ee

VA-DPO: Valence-Arousal Direct Preference Optimization for Controllable Emotion Generation in Language Models — arXiv:2608.20374

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.20374
License (as archived): CC0-1.0
Archived: 2026-10-18T10:18:12.846Z
SHA-256 of the archived text: 49de3747ce216d0d91f41cad0fc2c4d0de8a9f416eb7b168ba22df59d821514f
Stable URL of this page: https://metraipolis.com/documento/dc_66c1745524ee

How to cite

VA-DPO: Valence-Arousal Direct Preference Optimization for Controllable Emotion Generation in Language Models — arXiv:2608.20374 — arXiv (en), https://arxiv.org/abs/2608.20374 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_66c1745524ee (Metraipolis archive dc_66c1745524ee, sha256 49de3747ce216d0d91f41cad0fc2c4d0de8a9f416eb7b168ba22df59d821514f)

BibTeX

@misc{dc66c1745524ee,
  title = {VA-DPO: Valence-Arousal Direct Preference Optimization for Controllable Emotion Generation in Language Models — arXiv:2608.20374},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20374},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_66c1745524ee; sha256 49de3747ce216d0d91f41cad0fc2c4d0de8a9f416eb7b168ba22df59d821514f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20374. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

How precisely can we tell a language model how to feel? Most work on emotional generation answers with a discrete label - happy, angry, sad - which cannot express a target like "mildly downcast but calm." We instead specify the desired affect as a continuous point (v*, a*) in the Valence-Arousal plane and train the model to hit it. Our method, VA-DPO, is a small modification to Direct Preference Optimization: a frozen VA regressor scores each sampled generation by its Euclidean distance to the target, we keep only candidate pairs whose distance gap clears a margin tau, and we optimize a LoRA adapter with the ordinary DPO loss against a frozen reference. The DPO objective itself is unchanged; what is new is how the preference data is built. On Llama-3.1-8B-Instruct this cuts mean VA distance to the target by 33% over system-prompting and 25% over few-shot prompting, lifting valence/arousal correlation to r_v=0.93 and r_a=0.75. The gains carry over to Qwen3-8B and Llama-3.2-3B, and they do not come at the usual price: MMLU is unchanged (Delta=+0.0) and HellaSwag and TruthfulQA are preserved. We release the code, configs, and the preference-construction pipeline.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.20374 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:17:37.559Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).