Search papers, labs, and topics across Lattice.
This paper investigates the impact of dataset imbalance on auditory attention decoding (AAD) from EEG signals using stimulus reconstruction-based deep neural networks. They show that unbalanced datasets lead to overestimated decoding performance across three public EEG datasets (KUL, DTU, and NJU cEEGrid). To mitigate this, they introduce a leave-one-paired-envelope-out (LOPEO) cross-validation protocol that provides more accurate performance estimates on unbalanced AAD datasets.
Stimulus reconstruction-based auditory attention decoding from EEG signals is easily fooled: unbalanced datasets inflate decoding accuracy, but a new cross-validation method fixes this.
In the past decade, numerous studies have applied deep neural networks (DNNs) to decode auditory attention (AAD) from Electroencephalogram (EEG) signals via stimulus reconstruction. However, the influence of dataset balance on the decoding performance of stimulus reconstruction-based AAD remains unexplored. In this study, three publicly available EEG-AAD datasets - KUL, DTU, and NJU cEEGrid - are used to construct both balanced and unbalanced experimental conditions. We hypothesize and demonstrate that stimulus reconstruction-based DNN decoders tend to produce overestimated decoding performance on unbalanced datasets. To address this issue, we propose a leave-one-paired-envelope-out (LOPEO) cross-validation protocol. Experimental results confirm that LOPEO effectively prevents inflated decoding accuracy on unbalanced datasets. While balanced datasets are generally preferred in experimental design, LOPEO provides a principled evaluation framework for unbalanced datasets that have already been published, filling an important gap in the field.
