Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel method for generating super-resolution spectrograms by fusing multiple spectrograms with varying resolutions using optimal transport (OT) divergences. The approach computes a barycenter of input spectrograms, allowing for fusion even when the inputs don't share a time-frequency grid. A new transportation cost is introduced to preserve time-frequency geometry while reducing computational complexity, and an unbalanced OT framework with a block majorization-minimization algorithm is used for efficient computation.
Achieve super-resolution spectrograms by cleverly fusing multiple lower-resolution spectrograms with optimal transport, sidestepping the usual time-frequency uncertainty limitations.
Time-frequency representations, such as the short-time Fourier transform (STFT), are fundamental tools for analyzing non-stationary signals. However, their ability to achieve sharp localization in both time and frequency is inherently limited by the Gabor-Heisenberg uncertainty principle. In this paper, we address this limitation by introducing a method to generate super-resolution spectrograms through the fusion of two or more spectrograms with varying resolutions. Specifically, we compute the super-resolution spectrogram as the barycenter of input spectrograms using optimal transport (OT) divergences. Unlike existing fusion approaches, our method does not require the input spectrograms to share the same time-frequency grid. Instead, the input spectrograms can be computed using any STFT parameters, and the resulting super-resolution spectrogram can be defined on an arbitrary user-specified grid. We explore various OT divergences based on different transportation costs. Notably, we introduce a novel transportation cost that preserves time-frequency geometry while significantly reducing computational complexity compared to standard Wasserstein barycenters. We adopt the unbalanced OT framework and derive a new block majorization-minimization algorithm for efficient barycenter computation. We validate the proposed method on controlled synthetic signals and recorded speech using both quantitative and qualitative evaluations. The results show that our approach combines the best localization properties of the input spectrograms and outperforms an unsupervised state-of-the-art fusion method.