Search papers, labs, and topics across Lattice.
This paper introduces InterAligner, an innovative approach to training Aligner-Encoder based ASR models that enhances alignment learning by incorporating an intermediate Aligner objective and an intermediate CTC loss. The method addresses the brittleness of training on long utterances, which is a common challenge in current ASR systems. Experimental results on the LibriSpeech dataset demonstrate that InterAligner significantly reduces word error rates (WER) to 3.1/5.6, outperforming previous models, particularly on longer inputs.
InterAligner achieves a remarkable reduction in word error rates, especially for long utterances, by progressively forming alignment across the model's depth.
Aligner-Encoders are recently proposed seq2seq end-to-end ASR models that replace decoder attention by predicting the uth token directly from the u-th encoder position, so the encoder must learn the alignment internally without cross-attention or a transducer lattice. In practice, this alignment often forms abruptly in the upper layers, making training sensitive and brittle on long utterances. We propose InterAligner, which adds an intermediate Aligner objective so alignment can form progressively across depth, together with an intermediate CTC loss (InterCTC) to stabilize optimization. On LibriSpeech with a 17-layer Conformer, a final-only Aligner reaches 5.0/7.8 WER (test-clean/other). InterCTC improves to 3.4/6.0, and InterAligner further reduces WER to 3.1/5.6 with the largest gains on long utterances.
