Search papers, labs, and topics across Lattice.
This paper addresses the challenge of estimating MIDI velocities across different musical instruments, a critical aspect for expressive music generation and analysis. By leveraging a piano-trained velocity estimator and introducing differentiable SoundFont proxies (Diff-SFProxy), the authors effectively adapt velocity estimation to various instruments by focusing on note-wise acoustic parameters rather than waveform reconstruction. Experimental results demonstrate that Diff-SFProxy significantly outperforms traditional waveform-based methods, particularly in cross-instrument scenarios involving piano and guitar.
Differentiable SoundFont proxies can revolutionize MIDI velocity estimation, achieving superior performance across instruments by focusing on acoustic dynamics rather than raw waveforms.
Many music datasets contain MIDI notes but lack reliable velocities, defaulting to a constant value. This absence is especially problematic outside the piano domain, as velocity is a core component for expressive rendering, music generation, and performance analysis. This paper studies cross-instrument MIDI velocity estimation in this label-scarce setting. Starting from a piano-trained velocity estimator, we recast target-instrument adaptation as predicting renderer-conditioned velocities whose rendering matches the dynamics of the performance audio. This adaptation can be driven by either differentiable synthesizers (Diff-Synth) or our proposed differentiable SoundFont proxies (Diff-SFProxy). We highlight the Diff-SFProxy: it supervises velocity through note-wise, loudness-related acoustic parameters rather than waveform reconstruction, focusing gradients on velocity-dependent behavior. Experiments on piano and guitar show that Diff-SFProxy is effective for cross-instrument MIDI velocity estimation, while waveform-domain Diff-Synth degrades performance.
