Search papers, labs, and topics across Lattice.
This paper introduces LOUDAR, a novel music restoration method that operates in the latent space of a pretrained audio autoencoder, allowing for the recovery of clean audio signals from corrupted recordings without requiring paired training data. By modeling the unknown distortion as a learnable latent operator and leveraging an unconditional latent diffusion model for regularization, LOUDAR effectively adapts to various degradation types during inference. The method demonstrates significant improvements in restoring singing voices and guitar sounds, outperforming both supervised and unsupervised baselines in terms of audio quality.
LOUDAR adapts to unknown audio distortions on-the-fly, achieving superior restoration results without the need for paired training data.
Music restoration seeks to recover a clean signal from an observed recording degraded by an unknown effect, distortion, or corruption. Existing systems often rely on paired training data and distortion-specific supervision, which limits their use when the forward process is not known in advance. We propose LOUDAR (Latent-space Optimization of Unknown Distortion for Audio Restoration) a general-purpose restoration method that operates in the latent space of a pretrained audio autoencoder and models the unknown distortion as a learnable latent operator. At inference time, LOUDAR alternates between estimating the clean latent variable and updating the latent operator parameters. An unconditional latent diffusion model provides a prior over clean audio and regularizes this inference by steering the latent estimate toward the manifold of clean recordings. Because the degradation model is adapted per input, the approach is broadly applicable across diverse restoration problems. We evaluate LOUDAR on singing voice effect removal and restoration, as well as guitar distortion removal, and show that it consistently improves over degraded inputs and is competitive with supervised and unsupervised baselines in waveform and latent domains.
