Search papers, labs, and topics across Lattice.
This paper introduces Adaptive Multi-Teacher Relational Distillation (AMRD) to enhance on-device speech emotion recognition (SER) by addressing the challenges of teacher reliability and the limitations of logit-level distillation. By employing a one-class SVM to weight teachers based on logit similarity and incorporating a relational distillation loss, AMRD effectively captures inter-sample relationships that traditional methods overlook. Experimental results on IEMOCAP and CREMA-D datasets demonstrate that AMRD consistently outperforms single-teacher distillation approaches across various student architectures, highlighting the effectiveness of its dual-component strategy.
AMRD achieves superior performance in speech emotion recognition by leveraging the strengths of multiple teachers while capturing relational structures that traditional methods miss.
On-device speech emotion recognition (SER) is critical for real-time applications, yet large self-supervised models that excel at SER are too costly for edge devices. Multi-teacher knowledge distillation can compress them into a lightweight student, but two challenges remain: teacher reliability varies across batches, and logit-level distillation ignores inter-sample relational structure. We propose Adaptive Multi-teacher Relational Distillation (AMRD) to address both. A one-class SVM on each teacher's logit similarity matrix assigns per-batch weights favoring more coherent teachers. A relational distillation loss aligns teacher and student similarity matrices, capturing structure that logit matching misses. On IEMOCAP and CREMA-D datasets across four student architectures, AMRD outperforms single-teacher distillation baselines in most settings, and ablations confirm both components yield complementary gains.
