Search papers, labs, and topics across Lattice.
This paper introduces RelFx, a contrastive learning framework designed to learn relative audio effects transformations without the need for dry reference recordings, which are often impractical in real-world music production. By employing a dual-branch Siamese encoder with cross-attention and differential gating fusion, the model effectively infers shared effect transformations from pairs of audio clips, enhancing the representation of audio effects. The proposed method achieves state-of-the-art performance in audio effect style transfer, outperforming existing techniques across multiple instrument categories, thereby demonstrating the efficacy of relative effect distance in practical applications.
Learning audio effects without dry references not only boosts performance but also aligns better with real-world recording conditions.
Audio effects (Fx) representation learning plays a key role in intelligent music production, including automatic mixing and Fx style transfer. Existing methods typically rely on dry or nearly dry references for effect modeling, yet truly unprocessed audio is rarely available in practice, as real recordings inevitably reflect the microphone, room acoustics, and preceding signal processing. Instead of pursuing absolute effect encodings, we argue that the relative effect distance between audio signals is more meaningful for real-world music production. Motivated by this, we propose RelFx, a contrastive learning framework that learns relative effect transformations from general audio collections without requiring dry references during representation training. Our approach uses a dual-branch Siamese encoder equipped with cross-attention and differential gating fusion to infer the shared effect transformation from a reference clip and an effect-processed, content-related clip. We further propose an antisymmetric fusion variant for bidirectional effect encoding, such that swapping the input order directly produces a nearly sign-reversed embedding, a property not explored in earlier work. Moreover, our dry-reference-free formulation eliminates the reliance on dry multitrack datasets and enables training on effect-bearing audio. Experiments on Fx style transfer demonstrate state-of-the-art performance under the standard Fx-Encoder++ MUSDB18 evaluation protocol, consistently outperforming existing approaches across all four instrument categories.
