Search papers, labs, and topics across Lattice.
This paper introduces NeuroSonic, a conditional flow-matching framework designed to reconstruct speech from EEG signals by learning a deterministic probability-flow velocity field that transforms noise-corrupted acoustic states into clean speech. By embedding EEG and audio into a shared token space and utilizing a time-conditioned gated Transformer, NeuroSonic effectively models the evolution of speech trajectories without relying on stochastic sampling methods. Evaluations on the CineBrain and EAV benchmarks demonstrate significant improvements in distributional realism, spectral fidelity, and perceptual quality, particularly in challenging artifact-heavy segments, with a notable 26.3% increase in overall perceptual quality compared to existing methods.
NeuroSonic achieves a 26.3% improvement in perceptual quality for EEG-to-speech reconstruction, particularly excelling in artifact-heavy scenarios where traditional methods struggle.
Reconstructing continuous speech from scalp electroencephalography (EEG) remains fundamentally challenging. EEG provides a weak, spatially diffuse, and highly variable measurement of distributed cortical activity, whereas speech is organized as a coherent acoustic trajectory with strong harmonic and temporal structure. The resulting mismatch makes waveform regression unstable and causes stochastic multi-step generation to be sensitive to artifact-dependent conditioning and subject variability. We introduce NeuroSonic, a conditional flow-matching framework for EEG-to-speech reconstruction. Instead of predicting waveforms directly or refining them through stochastic denoising, NeuroSonic learns a deterministic probability-flow velocity field that transports a noise-corrupted acoustic state toward clean speech under EEG conditioning. EEG and audio are embedded into a shared token space and processed by a time-conditioned gated Transformer that parameterizes the transport ordinary differential equation. This formulation models trajectory evolution explicitly while avoiding iterative stochastic sampling. We evaluate NeuroSonic on the CineBrain and EAV benchmarks under cross-subject evaluation. Across both datasets, the proposed method improves distributional realism, spectral fidelity, and perceptual quality over representative GAN-, diffusion-, and mean-flow baselines, with up to a 26.3\% gain in overall perceptual quality. The performance gap is most evident in artifact-heavy segments, where conditioning variability is strongest. These findings indicate that deterministic conditional transport provides a stable and effective formulation for EEG-driven speech reconstruction. Code is available at https://github.com/Y-Research-SBU/NeuroSonic/ .
