Search papers, labs, and topics across Lattice.
This paper introduces a cross-modal framework that learns implicit music styles from raw audio to enhance symbolic music generation, specifically for piano arrangements. By employing a Querying Transformer to extract style representations from a pre-trained audio language model, the authors implement a two-stage training strategy that aligns auditory style with symbolic expression through contrastive learning and generative modeling. The results show significant advancements in style-aware alignment and music quality, particularly in piano cover generation and style transfer tasks.
Extracting implicit music styles from audio can dramatically enhance the quality and fidelity of symbolic music generation.
What is music style? Though often described using text labels such as"swing,""classical,"or"emotional,"the real style remains implicit and hidden in concrete music examples. In this paper, we introduce a cross-modal framework that learns implicit music styles from raw audio and applies them to symbolic music generation. Inspired by BLIP-2, our model leverages a Querying Transformer (Q-Former) to extract style representations from a large, pre-trained audio language model (LM), and further applies them to condition a symbolic LM for generating piano arrangements. We adopt a two-stage training strategy: contrastive learning to align auditory style with symbolic expression, followed by generative modeling for music arrangement. Our model generates piano performances jointly conditioned on a lead sheet (content) and a reference audio example (style), enabling controllable and stylistically faithful arrangement. Experiments demonstrate the effectiveness of our approach in piano cover generation, style transfer, and audio-to-MIDI retrieval, achieving substantial improvements in style-aware alignment and music quality.
