Search papers, labs, and topics across Lattice.
This paper introduces Neural Morphing, a novel method for controllable audio transformation using pretrained neural audio codecs, which leverages residual-vector-quantized (RVQ) token grains selected from a user-defined palette. By employing a continuity-constrained sequence matcher and an RVQ-group transfer policy, the approach ensures that the rhythmic structure of the source audio is maintained while allowing for flexible timbral modifications. The resulting system is implemented as a deployable VST3/AU plugin, demonstrating effective real-time audio processing capabilities.
Transform audio in real-time while preserving rhythm and enhancing timbre with user-selected token grains.
Neural audio codecs were originally developed for high-fidelity compression; however, their latent token representations and expressive decoders also constitute a powerful substrate for controllable audio transformation. This work introduces Neural Morphing, a training-free token-domain audio effect that selects residual-vector-quantized (RVQ) token grains from a user palette and decodes the edited stream through a pretrained codec. The method combines an RVQ-group transfer policy that separates coarse, middle, and fine codebook groups with a continuity-constrained sequence matcher that replaces independent greedy selection with bounded beam search. The intended output is a controlled hybrid: the source preserves rhythmic organization while the palette contributes timbral color and residual detail. We focus on the implementation and realtime behavior of a deployable VST3/AU system, including chunked rendering, palette-size scaling, and backend health checks.
