Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Echo, a novel audio system leveraging a single 25 M-parameter ViT encoder for simultaneous speaker diarization and speech recognition within a shared latent space. By employing a Joint-Embedding Predictive Architecture (JEPA) without per-task fine-tuning, Echo achieves impressive metrics on synthetic VoxCeleb2 mixtures, including a 15.00% blind Diarization Error Rate (DER) and 97.80% Permutation Invariant Training (PIT) separation accuracy. The findings highlight the potential of joint task learning in audio processing, despite not achieving state-of-the-art results for individual tasks, and reveal critical insights into the limitations of end-to-end automatic speech recognition (ASR) systems.
Achieving 15.00% blind DER and 97.80% PIT accuracy, Echo showcases the power of joint task learning in audio processing without task-specific fine-tuning.
We present Echo, a proof-of-concept audio system built around a single 25 M-parameter ViT encoder. The encoder is pretrained with a JEPA objective and then specialised by stages to carry speaker identity, phonetic content, and dynamic source routing in the same 512-dimensional latent space, with no per-task fine-tuning at deployment. Light heads handle diarization (ArcFace + VBx) and dynamic source separation (null-target K-set prediction). On synthetic VoxCeleb2 mixtures with unknown K, the canonical stack reaches 15.00% blind DER, 97.80% PIT separation accuracy with +9.52 dB latent SI-SDR, and a +53.50-point speaker/content factorisation gap on a held-out k-NN probe. The point of Echo is not a new SOTA on any single task but the joint coexistence of three tasks on one encoder at this footprint. We document the design stage by stage, report the dead-ends, and identify the structural wall on end-to-end ASR through the VQ bottleneck that still bounds the PoC.