Search papers, labs, and topics across Lattice.
This paper introduces Context-Aware Interleaved Batching, an innovative approach that enhances WhisperX's speech transcription by preserving historical context while maintaining high throughput. By leveraging Voice Activity Detection (VAD) to define segment boundaries, the method stabilizes text conditioning and mitigates issues related to context loss and hallucination in transcription. The results show a significant reduction in Word Error Rate (WER) and improved accuracy in proper noun transcription on long-form audio benchmarks, indicating a substantial advancement in transcription quality and efficiency.
Context-Aware Interleaved Batching cuts Word Error Rate while preserving context, revolutionizing how we handle speech transcription in real-time applications.
While WhisperX accelerates speech transcription via intra-audio batching, it isolates audio segments, losing the historical context needed for coherent punctuation and terminology transcription. Conversely, standard Whisper retains context sequentially but suffers from slow inference and hallucination loops. To achieve the best of both worlds, we propose Context-Aware Interleaved Batching. By using VAD-derived segment boundaries, our algorithm stabilizes Whisper's text conditioning, allowing us to safely maintain continuous historical context across batched audio segments. As demonstrated on long-form audio benchmarks, this approach reduces Word Error Rate (WER) and improves proper noun transcription, all while maintaining high-throughput inference speeds.
