Search papers, labs, and topics across Lattice.
This paper introduces PianoKontext, a flow matching rendering model designed to generate expressive piano performances from MIDI scores while overcoming the limitations of existing audio editing models that only handle synchronized music samples. By utilizing Dynamic Time Warping (DTW) in the latent space of a pretrained Music2Latent model, the authors construct paired training data that captures variable-length performance nuances. The key result demonstrates that PianoKontext effectively learns the dependencies between musical scores and expressive timing, enabling the generation of more realistic and varied piano performances.
Flow matching models can now generate expressive piano performances that capture nuanced timing, breaking free from the constraints of fixed-length audio samples.
Expressive performance rendering (EPR) aims to generate realistic performances constrained on sequences of notes. However, flow matching audio editing models manipulate only synchronized music samples of the same duration, limiting their understanding of expressive timing. We introduce PianoKontext, a flow matching rendering model for classical piano music that generates variable-length performances in the latent space of a pretrained Music2Latent model. We synthesize MIDI scores into deadpan audio and employ Dynamic Time Warping (DTW) in the latent space to construct paired data for training. The aligned embeddings are concatenated in DiT blocks, allowing for a simple and effective learning of the dependencies between the score and performances. Audio samples are available at our demo page: https://realfolkcode.github.io/pianokontext_demo/.
