Search papers, labs, and topics across Lattice.
TuneJury is an innovative instance-level pairwise reward model designed to enhance music generation by predicting preference scores based on text prompts and audio clips. The model is trained on a diverse set of human-preference labels and demonstrates strong calibration and generalization capabilities, outperforming previous baselines on out-of-distribution benchmarks. Notably, it introduces anchor calibration, allowing for efficient adaptation to new generators without the need for extensive retraining, thus improving performance across various downstream applications.
TuneJury achieves superior music preference alignment with a single frozen reward model that adapts efficiently to new audio generators.
We introduce TuneJury, an open, instance-level pairwise reward model for text-to-music that predicts a music preference score from a text prompt and an audio clip. The released checkpoint is trained on publicly available human-preference labels covering arena-style (A vs. B) votes, metric-alignment preference pairs, crowdsourced pairwise comparisons, and expert aesthetic ratings. The predicted score margin between two clips is well calibrated on our held-out test split, supporting data filtering via a simple score threshold. TuneJury generalizes to both held-out test pairs and out-of-distribution benchmarks, remaining competitive with prior baselines on the latter. For generators released after training, we introduce anchor calibration, a post-hoc, per-system Bradley-Terry calibration that recovers agreement at substantially better data efficiency than from-scratch retraining. The same frozen reward drives consistent reward-axis gains across three downstream applications: inference-time best-of-N selection, DITTO-style latent optimization, and expert-iteration post-training. TuneJury is available at https://github.com/yonghyunk1m/TuneJury.
CMU ML