Metraipolis › archive › dc_3cd25897baf7

SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue — arXiv:2609.01548

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.01548
License (as archived)
CC0-1.0
Archived
2026-10-17T06:41:38.803Z
SHA-256 of the archived text
1ad9059bc1e46cd38f991c1e041d0f2f3f480ad88684a12472df9705173c5b3e
Stable URL of this page
https://metraipolis.com/documento/dc_3cd25897baf7

How to cite

SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue — arXiv:2609.01548 — arXiv (en), https://arxiv.org/abs/2609.01548 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3cd25897baf7 (Metraipolis archive dc_3cd25897baf7, sha256 1ad9059bc1e46cd38f991c1e041d0f2f3f480ad88684a12472df9705173c5b3e)
BibTeX
@misc{dc3cd25897baf7,
  title = {SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue — arXiv:2609.01548},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.01548},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3cd25897baf7; sha256 1ad9059bc1e46cd38f991c1e041d0f2f3f480ad88684a12472df9705173c5b3e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01548. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) are increasingly used in advice seeking and decision making that may affect social judgements. Despite stigma's profound effects on people and communities, benchmarks remain scarce. Existing general-domain evaluations typically rely on static prompts and fixed-format tasks, overlooking conversational contexts and audience effects in everyday communication. To address these gaps, we introduce SDARE-Bench, the first scenario-based benchmark evaluating both stigma detection and open-ended response generation in LLMs, comprising 1,138 dyadic queries and 1,388 group dialogue. Empirical results across 8 LLMs consistently demonstrate poor identification of stigma components, especially in group dialogues. In open-ended response generation, stigma expression was substantially higher in group settings than in dyadic, with weaker resistance to stigma and more unrealistic advice. Responses were evaluated using a classifier trained on 1,392 human annotated responses. In constructed group pressure settings, stigma expression rates further increased to a striking average of 97.5%. Our findings identify stigma response as a recurring LLM safety vulnerability, especially in socially complex conversational contexts.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.01548 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:33:38.870Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).