Metraipolis › archive › dc_f47e0817cb67

RELATE: An Evaluation Framework for measuring Relational Orientation of Large Language Models — arXiv:2610.09569

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.09569
License (as archived)
CC0-1.0
Archived
2026-10-08T11:26:10.064Z
SHA-256 of the archived text
f23b22216431f796b1eec9f82ad78f6852406b07bd43f35b6f4d30a2f0e759d1
Stable URL of this page
https://metraipolis.com/documento/dc_f47e0817cb67

How to cite

RELATE: An Evaluation Framework for measuring Relational Orientation of Large Language Models — arXiv:2610.09569 — arXiv (en), https://arxiv.org/abs/2610.09569 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f47e0817cb67 (Metraipolis archive dc_f47e0817cb67, sha256 f23b22216431f796b1eec9f82ad78f6852406b07bd43f35b6f4d30a2f0e759d1)
BibTeX
@misc{dcf47e0817cb67,
  title = {RELATE: An Evaluation Framework for measuring Relational Orientation of Large Language Models — arXiv:2610.09569},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.09569},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f47e0817cb67; sha256 f23b22216431f796b1eec9f82ad78f6852406b07bd43f35b6f4d30a2f0e759d1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09569. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used for emotional support, raising concern that sustained use may draw users away from their real-world relationships. Yet existing evaluations primarily focus on the safety, empathy, or helpfulness of responses, leaving under-examined a relational question: where does the model orient the user for continued support? To address this question, we introduce relational orientation, a property operationalized through two non-exclusive dimensions: inward-facing (IF) language, which positions the AI as the user's ongoing source of support, and outward-scaffolding (OS) language, which encourages real-world human connection. Grounded in psychological and sociological literature, we formalize a taxonomy of relational orientation and present RELATE, a persona-conditioned framework for measuring inward-facing and outward-scaffolding language at the sentence level in multi-turn dialogues. RELATE pairs 76 help-seeking situations adapted from naturally occurring questions with three simulated user styles, providing 228 evaluation stimuli. In our experiments, we evaluate seven LLMs using dialogues with six assistant turns each, yielding 1,596 dialogues and 69,194 assistant sentences. We assess these sentences using a primary rubric-based LLM judge and apply a secondary judge to a subset. Under automated evaluation, we find that the proportion of sentences labeled as IF is higher at the sixth assistant turn than at the first, while the proportion labeled as OS is substantially lower for hesitant, indirect simulated users than for explicit, reassurance-seeking users. RELATE provides a reproducible framework and a sentence-level signal for auditing and steering the relational orientation of supportive LLMs.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.09569 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:24:06.323Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).