Search papers, labs, and topics across Lattice.
This paper systematically evaluates various sample-level tokenization strategies, including a novel autoencoder-based learnable tokenizer, for MEG data used in transformer-based large neuroimaging models (LNMs). The study assesses tokenizers based on signal reconstruction fidelity, token prediction, biological plausibility, subject-specific information preservation, and downstream task performance. Results indicate that both learnable and non-learnable tokenization methods achieve high reconstruction accuracy and comparable performance, suggesting the viability of simple fixed sample-level tokenization for neural foundation models.
Forget complex learned tokenizers for MEG data – simple, fixed sample-level tokenization works just as well for foundation models.
Recent success in natural language processing has motivated growing interest in large-scale foundation models for neuroimaging data. Such models often require discretization of continuous neural time series data, a process referred to as 'tokenization'. However, the impact of different tokenization strategies for neural data is currently poorly understood. In this work, we present a systematic evaluation of sample-level tokenization strategies for transformer-based large neuroimaging models (LNMs) applied to magnetoencephalography (MEG) data. We compare learnable and non-learnable tokenizers by examining their signal reconstruction fidelity and their impact on subsequent foundation modeling performance (token prediction, biological plausibility of generated data, preservation of subject-specific information, and performance on downstream tasks). For the learnable tokenizer, we introduce a novel approach based on an autoencoder. Experiments were conducted on three publicly available MEG datasets spanning different acquisition sites, scanners, and experimental paradigms. Our results show that both learnable and non-learnable discretization schemes achieve high reconstruction accuracy and broadly comparable performance across most evaluation criteria, suggesting that simple fixed sample-level tokenization strategies can be used in the development of neural foundation models. The code is available at https://github.com/OHBA-analysis/Cho2026_Tokenizer.
