Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study addresses the complex challenge of predicting student engagement in online tutoring videos by developing a multimodal framework that integrates various behavioral and contextual cues, including spatiotemporal features from video, audio, and image encoders. The authors leverage a Perceiver IO architecture to combine these modalities and model student and instructor personalities as variational posteriors, enhancing the framework's adaptability across different participants. Their approach not only achieves competitive performance on the challenging CASED dataset but also provides well-calibrated uncertainty metrics, highlighting the importance of risk quantification for real-world educational applications.
Engagement prediction in online learning is not just about accuracy; it requires robust uncertainty quantification to ensure effective real-world application.
The prediction of student engagement from the online tutoring videos is difficult because engagement is a multidimensional construct comprising distinct behavioral, emotional, and cognitive states. A reliable prediction requires bringing together different types of behavioral signals as well as expressive cues. Through our analysis of the CASED dataset, it is clear that engagement prediction gets even harder due to the high inter-person variability as well as the subjectivity of the engagement annotation. To tackle these challenges, we develop a multimodal framework that integrates the implicit spatiotemporal features extracted from pretrained video, audio, and image encoders along with structured behavioral modalities like head pose, gaze, facial action units, emotion, and wavelet-based audio features. We integrate these modalities via a Perceiver IO latent bottleneck. Moreover, student and instructor personalities are modeled as variational posteriors over learnable embeddings to enable partial pooling across participants. We employ evidential regression and spectral-normalized Gaussian process classification heads for uncertainty-aware prediction to further improve robustness and calibration. Benchmark on the CASED challenge test set shows that all participating methods converge near random-chance performance, revealing the difficulty of the dataset. In this highly ambiguous regime, our framework achieves competitive performance while uniquely offering well-calibrated uncertainty metrics, demonstrating that reliable risk-quantification is an essential prerequisite for deploying engagement models in real-world educational tools.