Search papers, labs, and topics across Lattice.
The paper addresses the problem of reconstructing speech envelopes from EEG signals, which is crucial for neuro-steered hearing aids. They introduce a dynamic state-space fusion model (DECAF) that combines direct EEG-based estimates with predictions from recent speech context, using a learned gating mechanism for adaptive cue balancing. Evaluation on the ICASSP 2023 Stimulus Reconstruction benchmark demonstrates significant improvements over static EEG-only baselines, highlighting the synergy between neural and temporal information.
By dynamically fusing EEG data with a predictive temporal prior derived from recent speech context, DECAF significantly boosts the accuracy of speech envelope reconstruction, outperforming static methods.
Reconstructing the speech audio envelope from scalp neural recordings (EEG) is a central task for decoding a listener's attentional focus in applications like neuro-steered hearing aids. Current methods for this reconstruction, however, face challenges with fidelity and noise. Prevailing approaches treat it as a static regression problem, processing each EEG window in isolation and ignoring the rich temporal structure inherent in continuous speech. This study introduces a new, dynamic framework for envelope reconstruction that leverages this structure as a predictive temporal prior. We propose a state-space fusion model that combines direct neural estimates from EEG with predictions from recent speech context, using a learned gating mechanism to adaptively balance these cues. To validate this approach, we evaluate our model on the ICASSP 2023 Stimulus Reconstruction benchmark demonstrating significant improvements over static, EEG-only baselines. Our analyses reveal a powerful synergy between the neural and temporal information streams. Ultimately, this work reframes envelope reconstruction not as a simple mapping, but as a dynamic state-estimation problem, opening a new direction for developing more accurate and coherent neural decoding systems.
