TuneJury: An Open Metric for Improving Music Generation Preference Alignment

2026-06-15

AI Summary

This work addresses the lack of open, reliable evaluation metrics for human preference alignment in text-to-music generation by proposing an open-source, instance-level pairwise reward model. Built upon the Bradley–Terry framework, the model integrates multi-source human preference data and incorporates an anchor-based calibration mechanism, enabling improved cross-system preference consistency without retraining or updating model parameters. The approach demonstrates strong performance on both in-distribution and out-of-distribution benchmarks, significantly enhancing downstream tasks such as best-sample selection, latent-space guidance, and expert iterative training. By offering a generalizable, efficient, and practical tool for preference-aligned evaluation and optimization, this method advances the development of music generation systems that better reflect human judgments.

Abstract

We introduce TuneJury, an open, instance-level pairwise reward model for text-to-music that predicts a music preference score from a text prompt and an audio clip. The released checkpoint is trained on publicly available human-preference labels covering arena-style (A vs. B) votes, metric-alignment preference pairs, crowdsourced pairwise comparisons, and expert aesthetic ratings. The predicted score margin between two clips is well calibrated on our held-out test split, supporting data filtering via a simple score threshold. TuneJury generalizes to both held-out test pairs and out-of-distribution benchmarks, remaining competitive with prior baselines on the latter. For generators released after training, we introduce anchor calibration, a post-hoc, per-system Bradley-Terry calibration that recovers agreement at substantially better data efficiency than from-scratch retraining. The same frozen reward drives consistent reward-axis gains across three downstream applications: inference-time best-of-N selection, DITTO-style latent optimization, and expert-iteration post-training. TuneJury is available at https://github.com/yonghyunk1m/TuneJury.

Problem

• music generation
• preference alignment
• reward model
• text-to-music
• human preference

Innovation

• preference alignment
• reward modeling
• music generation
• anchor calibration
• Bradley-Terry model