Search papers, labs, and topics across Lattice.
This paper investigates cross-domain transfer learning for predicting expert impressions of counseling quality, finding that training on out-of-domain simulated counseling conversations significantly outperforms in-domain training. The key to this transferability lies in using construct scores derived from expert rating instruments, extracted by small open-weight LLMs from transcripts, which capture session-level quality better than generic dialogue features. The research demonstrates that language-based features, particularly those aligned with expert assessment instruments, are robust predictors of counseling quality across different simulated domains, even with limited data.
Expert counseling quality assessment transfers surprisingly well across domains, with LLM-extracted instrument-based constructs outperforming in-domain training and even beating generic dialogue features.
Automatic assessment of communication quality in dyadic counseling conversations is bottlenecked by data: expert-rated corpora are small and expensive to grow. We study cross-domain transfer of expert overall-impression prediction across three German corpora of simulated counseling (two general-practice medical, one school-related parent-teacher; $n=195$ expert-rated sessions, one corpus after scale equating). Training on the other domains beats training in-domain: leave-one-domain-out transfer reaches nested Spearman $\rho = 0.54$ against $\le 0.48$ within the target domain, a paired session-level gap of $+0.15$ that holds at $+0.12$ when the training-set sizes are matched, so it is not simply data volume. The decisive features are session-level construct scores from small open-weight LLMs reading the two-speaker transcript, with the constructs largely derived from the experts'rating instruments: the instrument-derived battery lifts a single judge from $0.32$ to $0.41$ over generic dialogue qualities, judges from three model families ensemble to $0.51$ language-only, and a nonverbal-dyadic block adds $+0.03$ more, not separable from noise at this sample size. We also price the recording setup: one corpus lost its per-speaker audio, 16% of its diarised segments carry the wrong speaker, and repair is worth $+0.07$ there. At practically attainable corpus sizes, the expert's overall impression is carried by what is said, and by other communication programs'data more than by one's own.
