Search papers, labs, and topics across Lattice.
This paper introduces SUP-MCRL, a novel framework designed to enhance neural visual decoding in non-invasive brain-computer interfaces by addressing the limitations of traditional multimodal contrastive representation learning. By integrating a Semantic-entity Aware Visual Encoder, a Unified EEG Enhancer, and a Prototype-based Progressive Augmenter, the approach improves both semantic consistency and subject selectivity, leading to significant gains in zero-shot accuracy. The results show that SUP-MCRL achieves 66.0% Top-1 and 91.9% Top-5 intra-subject accuracy, as well as 24.0% and 52.9% accuracy in leave-one-subject-out (LOSO) settings, outperforming existing methods in the field.
Achieving a remarkable 66% accuracy in zero-shot EEG visual decoding could redefine benchmarks in brain-computer interface fidelity.
Non-invasive brain-computer interfaces suffer severe fidelity degradation in neural visual decoding when generalizing to natural visual experiences. Conventional multimodal contrastive representation learning solely optimizes geometric distance alignment, neglecting semantic consistency and subject selectivity, causing spurious zero-shot alignment. We propose SUP-MCRL, a unified framework integrating three collaborative mechanisms: (1) Semantic-entity Aware Visual Encoder (SAVE), learning spatial attention to extract semantic content without pre-trained saliency models; (2 Unified EEG Enhancer (UEE), employing multi-scale atrous convolutions and inter-band attention for adaptive cross-subject robustness; and (3) Prototype-based Progressive Augmenter (PPA), maintaining an EMA-updated pseudo-feature pool to prevent representation collapse. Zero-shot experiments on THINGS-EEG achieve 66.0%/91.9% (Top-1/Top-5) intra-subject and 24.0%/52.9% LOSO accuracy, surpassing state-of-the-art methods. Code is available at https://github.com/NZWANG/SUP-MCRL.
