Search papers, labs, and topics across Lattice.
OLIVE introduces a self-supervised framework for speech representation learning that integrates view-augmented masked latent prediction with waveform reconstruction. This dual-objective approach ensures that early encoder features preserve signal-level information while enhancing later contextual representations for improved robustness. The results indicate that OLIVE significantly enhances performance across various tasks, particularly in generation and speaker identification, while maintaining competitive results in recognition and semantic tasks.
OLIVE achieves superior speech representation that boosts generation and speaker tasks while preserving competitive recognition performance through innovative dual-objective training.
We propose Online Latent prediction with Invariant Views and rEconstruction (OLIVE), a self-supervised speech representation learning framework that jointly optimizes analysis and synthesis objectives. OLIVE combines view-augmented masked latent prediction with waveform reconstruction under a unified objective. Reconstruction constrains early encoder features to retain signal-level information, while masked latent prediction shapes later contextual representations toward invariance for robust downstream performance. We show that these objectives enable representations that support a broad range of tasks. In particular, OLIVE improves results on generation and speaker tasks, maintains competitive performance on recognition and semantic tasks, and improves waveform reconstruction.
