Search papers, labs, and topics across Lattice.
The paper introduces CaMBRAIN, a causal Mamba-based state space model for real-time EEG inference, addressing the quadratic scaling issues of attention-based models and the limitations of fixed-length input requirements in processing long EEG signals. To train CaMBRAIN, the authors develop a multi-stage self-supervised training pipeline that promotes long-range memory retention in the hidden state, crucial for streaming inference. CaMBRAIN achieves state-of-the-art results on three EEG datasets with over 10x higher throughput, enabling continuous, variable-length EEG signal inference.
CaMBRAIN unlocks real-time, continuous EEG analysis with a Mamba-based architecture, finally making long-range brain activity monitoring practical.
Electroencephalography (EEG) is a critical, non-invasive method to monitor electrical brain activity. EEGs can span anywhere from a couple seconds to multiple hours, posing a major hurdle for existing deep learning methods due to two major factors: (1) existing EEG models are predominantly built upon the attention mechanism, incurring quadratic scaling as the sequence length increases, and (2) raw EEG signals must be processed in a sliding-window fashion due to fixed-length input requirements, preventing global understanding of the entire signal. To this extent, we propose CaMBRAIN - the first Causal, Mamba-based state space model (SSM) capable of real-time inference of EEG signals, arguing that bidirectional approaches are needlessly expensive given the causal, unidirectional nature of EEG. However, training such a model is non-trivial, as crucial EEG events can be extremely brief - within fractions of a second - yet separated by long intervals spanning minutes. Current EEG methods use self-supervised objectives that optimize for signal reconstruction, but these are not well suited for streaming SSMs; they fail to explicitly train the hidden state to retain the salient long-range context needed for streaming inference. We therefore introduce a multi-stage self-supervised training pipeline specifically tailored to encourage long-range memory retention and strong performance on EEG signals, while preserving the linear-time complexity of state space models. CaMBRAIN achieves state-of-the-art (SOTA) results across 3 different EEG datasets with >10x higher throughput than existing models, enabling the first model capable of long-range, continuous inference of variable-length EEG signals.
