Search papers, labs, and topics across Lattice.
This paper introduces MOJO, a novel training framework that combines self-supervised learning (SSL) with supervised learning (SL) to enhance spike-tokenizing models for neural population decoding. By leveraging unlabelled data, MOJO demonstrates superior performance on multiple spiking datasets, particularly in scenarios with limited labelled data, such as few-shot finetuning. The framework not only improves decoding accuracy but also yields more interpretable neuronal representations, showcasing its efficacy across different neural modalities, including human electrocorticography.
MOJO outperforms traditional supervised learning models by effectively utilizing unlabelled data, achieving state-of-the-art results in neural decoding tasks with minimal labelled input.
Robust and accurate neural decoders are integral to neurotechnologies such as brain-computer interfaces and closed-loop experiments. Recent work has shown that tokenizing neural data at the spike level facilitates multi-session pretraining and delivers state-of-the-art decoding performance. However, current spike-based models are restricted to supervised learning (SL), limiting training to datasets with paired behavioural labels. To address this limitation, we introduce MOJO (Masked autOencoder-based JOint training), a training framework for spike-tokenizing models that jointly leverages self-supervised learning (SSL) via masked autoencoding and SL objectives. We evaluate MOJO on three spiking datasets spanning monkey motor cortex during reaching tasks and multi-regional mouse recordings during vision and decision making tasks, demonstrating superior performance over purely SL-trained models. This improvement is especially pronounced when training with limited labelled data, particularly in few-shot finetuning, where only a small amount of labelled data from a new session is available. Incorporating SSL also yields more interpretable neuronal representations, improving performance on brain region classification and spike-statistics prediction without explicit optimization for these tasks. We further show that MOJO generalizes beyond spiking data to human electrocorticography during speech, where it continues to outperform purely SL-trained models and achieves performance comparable to neuro-foundation models (NFMs) designed specifically for continuous signals. Overall, augmenting spike-tokenizing models with SSL improves performance in label-impoverished settings and enables the use of unlabelled data across various tasks and species, while generalizing to other neural modalities. These results suggest a path towards more flexible and scalable data usage when training NFMs.
Mila