Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study develops a framework for detecting speaker confidence by combining traditional speech features with embeddings from the Whisper encoder, addressing the challenge of limited labeled data through a pseudo-labelling technique. By integrating human-engineered features such as pitch, volume, and speech rate with model-generated labels, the approach achieves an overall accuracy of 75%. The findings underscore the potential for enhanced personalized feedback in educational settings, ultimately improving learning outcomes.
Achieving 75% accuracy in speaker confidence detection by fusing traditional features with Whisper embeddings could revolutionize personalized feedback in education.
Understanding speaker confidence is crucial in educational settings, as it can enhance personalised feedback and improve learning outcomes. This study introduces a novel framework for detecting speaker confidence by integrating human-engineered features with embeddings from the Whisper encoder. To address data limitations, a pseudo-labelling technique is employed to expand the labelled dataset, allowing the model to learn from both human-annotated and model-generated labels. The framework combines traditional speech features including pitch, volume, rate of speech, and the presence of disfluencies and stress, with Whisper embeddings, and uses a co-attention mechanism to fuse these representations and achieve an overall accuracy of 75%. This study contributes to advancing speech analysis, enabling applications that support personalised learning and speaking skill development.