Metraipolis › archive › dc_ffd08519f0f1
Language Carries the Expert's Impression: Instrument-Anchored LLM Judges Transfer Counseling-Quality Assessment and Beat In-Domain Training — arXiv:2610.08055
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.08055
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T11:23:19.962Z
- SHA-256 of the archived text
0688667a31c18c0519e9fb56a2dff5cb535b88f9cb74993f99e901913be68ab1- Stable URL of this page
https://metraipolis.com/documento/dc_ffd08519f0f1
How to cite
Language Carries the Expert's Impression: Instrument-Anchored LLM Judges Transfer Counseling-Quality Assessment and Beat In-Domain Training — arXiv:2610.08055 — arXiv (en), https://arxiv.org/abs/2610.08055 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffd08519f0f1 (Metraipolis archive dc_ffd08519f0f1, sha256 0688667a31c18c0519e9fb56a2dff5cb535b88f9cb74993f99e901913be68ab1)
BibTeX
@misc{dcffd08519f0f1,
title = {Language Carries the Expert's Impression: Instrument-Anchored LLM Judges Transfer Counseling-Quality Assessment and Beat In-Domain Training — arXiv:2610.08055},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.08055},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffd08519f0f1; sha256 0688667a31c18c0519e9fb56a2dff5cb535b88f9cb74993f99e901913be68ab1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08055. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Automatic assessment of communication quality in dyadic counseling conversations is bottlenecked by data: expert-rated corpora are small and expensive to grow. We study cross-domain transfer of expert overall-impression prediction across three German corpora of simulated counseling (two general-practice medical, one school-related parent-teacher; $n=195$ expert-rated sessions, one corpus after scale equating). Training on the other domains beats training in-domain: leave-one-domain-out transfer reaches nested Spearman $\rho = 0.54$ against $\le 0.48$ within the target domain, a paired session-level gap of $+0.15$ that holds at $+0.12$ when the training-set sizes are matched, so it is not simply data volume. The decisive features are session-level construct scores from small open-weight LLMs reading the two-speaker transcript, with the constructs largely derived from the experts' rating instruments: the instrument-derived battery lifts a single judge from $0.32$ to $0.41$ over generic dialogue qualities, judges from three model families ensemble to $0.51$ language-only, and a nonverbal-dyadic block adds $+0.03$ more, not separable from noise at this sample size. We also price the recording setup: one corpus lost its per-speaker audio, 16% of its diarised segments carry the wrong speaker, and repair is worth $+0.07$ there. At practically attainable corpus sizes, the expert's overall impression is carried by what is said, and by other communication programs' data more than by one's own. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.08055 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T11:15:39.446Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).