Search papers, labs, and topics across Lattice.
This paper introduces SiamCTC, a novel framework that combines Siamese networks with Connectionist Temporal Classification (CTC) to learn speech representations without the need for strict frame-level alignment. By allowing flexible, monotonic alignments between different temporal realizations of the same content, SiamCTC effectively accommodates variations in speaking rates and enhances robustness in downstream tasks. Experimental results indicate that SiamCTC significantly improves adaptability in speech representations, particularly when dealing with diverse speaking styles and rates.
Flexible alignments in speech representation learning can boost adaptability across varying speaking rates, challenging the need for strict frame-wise correspondence.
Self-supervised speech representation learning has made significant progress through Siamese networks, which leverage different views of the same input. However, existing methods often require frame-wise alignment between these views, overlooking the broader linguistic context invariance across different speaking styles. We introduce SiamCTC, a framework that integrates Siamese networks with Connectionist Temporal Classification (CTC) to learn speech representations without strict frame-level correspondence. By employing CTC loss to establish flexible, monotonic alignments between differing temporal realizations of the same content, SiamCTC accommodates speed perturbations and other temporal augmentations. This design relaxes frame-wise constraints while preserving temporal coherence and enhancing robustness to speaking-rate variations in downstream tasks. Our experiments demonstrate that SiamCTC leads to more adaptable speech representations, particularly at diverse speaking rates.
