Search papers, labs, and topics across Lattice.
This paper introduces GC-LoRA, a novel adapter architecture that enhances Parameter Efficient Fine-Tuning (PEFT) methods by incorporating Conformer-style local convolutional processing into pretrained Transformer encoders. The approach addresses the performance degradation of speech models in acoustically mismatched domains by effectively capturing local acoustic dependencies while preserving global representations. Experimental results show that GC-LoRA achieves significant Word Error Rate (WER) reductions of up to 10.9% across various challenging datasets, demonstrating its efficiency in parameter usage.
GC-LoRA reduces Word Error Rates by up to 10.9% in challenging acoustic environments while adding minimal trainable parameters.
Transformer-based Speech Foundation Models excel in most Automatic Speech Recognition tasks but often suffer performance degradation when applied to domains with mismatched acoustic characteristics. While Parameter Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), adjust global attention, they lack the local context modeling crucial for capturing domain-specific variations. We propose GC-LoRA, a novel adapter architecture that injects Conformer-style local convolutional processing into pretrained Transformer encoders. By integrating a lightweight adapter to encoder attention output projections, our method efficiently captures local acoustic dependencies without disrupting pretrained global representations. Experiments across diverse datasets (acoustically-degraded, bandlimited, dialectal, child) demonstrate the efficacy of our approach, achieving Word Error Rate (WER) reductions of up to 10.9% compared to baselines while adding minimal trainable parameters.
