Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces a multi-branch deep learning framework for detecting Parkinson's disease (PD) through speech analysis, leveraging three distinct speech representation modalities: Log-Mel spectrograms, MFCCs, and HuBERT embeddings. By employing a context-guided cross-modal attention mechanism, the model dynamically integrates these heterogeneous representations, achieving impressive performance metrics with an accuracy of 91.51%, F1-score of 91.24%, and AUC of 95.97% on the Spanish PC-GITA corpus. The findings underscore the importance of utilizing diverse speech features to enhance the robustness and clinical reliability of PD detection methods.
A context-guided cross-modal attention mechanism boosts Parkinson's disease detection accuracy by integrating diverse speech representations, achieving over 91% accuracy.
Parkinson's disease (PD) is a progressive neurodegenerative disorder that frequently causes speech impairments associated with hypokinetic dysarthria. As speech production relies on the precise coordination of complex neuromuscular mechanisms, speech analysis has emerged as a promising non-invasive and cost-effective biomarker for early PD detection. Recent deep learning approaches have shown encouraging results; however, most existing methods rely on a single speech representation, potentially overlooking complementary pathological information encoded across different feature spaces. In this work, we propose a multi-branch deep learning framework for automatic PD detection from speech. Each recording is segmented into 5-second chunks and represented using three complementary modalities: Log-Mel spectrograms, MFCCs, and HuBERT embeddings extracted from raw waveforms. The spectrograms are processed using a pre-trained ResNet-18 encoder, MFCC sequences are modeled through a BiLSTM network, and raw speech is encoded using a pre-trained HuBERT model. To effectively integrate these heterogeneous representations, we introduce a context-guided cross-modal attention mechanism that dynamically weights temporal HuBERT embeddings according to the global acoustic context derived from the spectrogram and MFCC branches. Experiments conducted on the publicly available Spanish PC-GITA corpus under strict speaker-independent 5-fold cross-validation demonstrate the effectiveness of the proposed approach. The proposed architecture achieves an accuracy of 91.51%, an F1-score of 91.24%, and an AUC of 95.97%. Furthermore, ablation studies confirm the contribution of both the proposed context-guided cross-modal attention mechanism and the integration of complementary speech representations. These findings highlight the potential of heterogeneous speech modeling for robust and clinically reliable PD detection.