Search papers, labs, and topics across Lattice.
LuMamba, a novel self-supervised framework, addresses the challenges of varying electrode topologies and computational scalability in EEG modeling by combining topology-invariant encodings using LUNA's learned-query cross-attention with efficient temporal modeling via FEMBA's bidirectional Mamba blocks. The framework introduces a systematic investigation of the Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA) for biosignal learning, finding that a combination of masked reconstruction and LeJEPA objectives yields the most robust performance. Pre-trained on 21,000 hours of EEG data, LuMamba achieves state-of-the-art performance on Alzheimer's detection (0.97 AUPR) and 80.99% balanced accuracy on TUAB, while requiring 377x fewer FLOPS and scaling to 12x longer sequences compared to existing methods.
Foundation models for EEG can now be 377x more efficient and handle 12x longer sequences, thanks to a novel Mamba-based architecture that also cracks the code for handling variable electrode setups.
Electroencephalography (EEG) enables non-invasive monitoring of brain activity across clinical and neurotechnology applications, yet building foundation models for EEG remains challenging due to \emph{differing electrode topologies} and \emph{computational scalability}, as Transformer architectures incur quadratic sequence complexity. As a joint solution, we propose \textbf{LuMamba} (\textbf{L}atent \textbf{U}nified \textbf{Mamba}), a self-supervised framework combining topology-invariant encodings with linear-complexity state-space modeling, using LUNA's learned-query cross-attention mechanism for channel unification~\cite{luna}, and FEMBA's bidirectional Mamba blocks for efficient temporal modeling~\cite{femba}. Within this architecture, we provide the first systematic investigation of the Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA) for biosignal learning. Pre-trained on over 21,000 hours of unlabeled EEG from the TUEG corpus, LuMamba is evaluated on five downstream tasks spanning abnormality detection, artifact recognition, and mental condition classification across electrode configurations ranging from 16 to 26 channels. In the pre-training objective, masked reconstruction alone yields structured but less generalizable representations, while LeJEPA alone produces diffuse embeddings; combining both objectives achieves the most robust performance. With only 4.6M parameters, LuMamba attains 80.99\% balanced accuracy on TUAB and achieves state-of-art performance on Alzheimer's detection (0.97 AUPR), while requiring \textbf{377$\times$ fewer FLOPS} than state-of-art models at equivalent sequence lengths and scaling to \textbf{12$\times$ longer sequences} before reaching typical GPU memory limits. Code is available at https://github.com/pulp-bio/biofoundation
