Search papers, labs, and topics across Lattice.
The paper introduces Latent Fourier Transform (LatentFT), a diffusion autoencoder framework that incorporates a latent-space Fourier transform to enable frequency-domain control over generative music models. By masking latent frequencies during training, LatentFT achieves disentangled representations that allow for coherent manipulation of musical structure at different timescales during inference. Experiments demonstrate that LatentFT improves condition adherence and generation quality compared to baselines, while also providing insights into the latent space organization of musical attributes.
Control the groove: a latent-space Fourier transform lets you remix and blend musical styles by directly manipulating the frequency components of musical structure.
We introduce the Latent Fourier Transform (LatentFT), a framework that provides novel frequency-domain controls for generative music models. LatentFT combines a diffusion autoencoder with a latent-space Fourier transform to separate musical patterns by timescale. By masking latents in the frequency domain during training, our method yields representations that can be manipulated coherently at inference. This allows us to generate musical variations and blends from reference examples while preserving characteristics at desired timescales, which are specified as frequencies in the latent space. LatentFT parallels the role of the equalizer in music production: while traditional equalizers operates on audible frequencies to shape timbre, LatentFT operates on latent-space frequencies to shape musical structure. Experiments and listening tests show that LatentFT improves condition adherence and quality compared to baselines. We also present a technique for hearing frequencies in the latent space in isolation, and show different musical attributes reside in different regions of the latent spectrum. Our results show how frequency-domain control in latent space provides an intuitive, continuous frequency axis for conditioning and blending, advancing us toward more interpretable and interactive generative music models.
MIT CSAIL