UT-AISTimprt submission for ICME 2026 Grand Challenge on Academic Text-to-Music Generation

2026-07-01

AI Summary

This work addresses the susceptibility of text-to-music generation to gradient interference under low-data and small-model regimes. To mitigate this issue, the authors propose clustering the training data based on either textual or audio embeddings and grouping semantically similar samples into the same batch. The study presents the first systematic comparison of clustering efficacy between the two modalities and elucidates how cluster granularity influences generation quality: clustering with text embeddings yields superior objective metrics, while a moderate number of clusters maximizes these scores and finer-grained clustering enhances musical structural coherence. By integrating embedding-based clustering, a tailored batch sampling strategy, and multi-dimensional evaluation, this research offers a novel and efficient approach to text-to-music generation.

Abstract

This work investigates the effect of batch sampling strategies during training for text-to-audio music generation under low-data and small-scale model settings. This paper describes our approach and findings for the ICME 2026 Grand Challenge on Academic Text-to-Music Generation. Training data are clustered using either text embeddings or audio embeddings, and samples with similar characteristics are grouped within the same mini-batch to mitigate gradient interference. The effects of modality and cluster granularity on clustering are analyzed. Results show that clustering based on text embeddings achieves better performance on objective evaluation metrics than clustering based on audio embeddings. In addition, different cluster granularity leads to different behaviors across evaluation criteria: a moderate number of clusters performs best on objective metrics, while a larger number of clusters tends to exhibit music with more coherent structure in listening tests.

Problem

• text-to-music generation
• batch sampling
• low-data setting
• small-scale model
• training strategy

Innovation

• batch sampling
• text-to-music generation
• embedding clustering
• gradient interference mitigation
• cluster granularity