MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling

2026-06-29

AI Summary

This work addresses the challenge of text-driven singing voice editing, where preserving the original melody, total duration, and unedited regions simultaneously is critical. The proposed MeloDISinger model achieves melody-aware and duration-preserving editing through audio inpainting, featuring three key innovations: a fixed-budget duration ratio prediction mechanism (MeloDRP), a cross-attention module that fuses phoneme and pseudo-MIDI melody context, and a soft phoneme–note alignment derived from time-overlapped supervision. Integrated with a flow-matching vocoder and an evaluation pipeline leveraging WhisperX and large language models, MeloDISinger achieves state-of-the-art performance in both objective metrics and subjective listening tests, significantly enhancing melodic consistency and duration fidelity after lyric editing.

Abstract

Text-based singing voice editing (SVE) aims to revise sung lyrics while preserving the original melody, total duration, and non-edited regions. In this paper, we propose MeloDISinger, a flow-matching-based SVE model for melody-aware and duration-preserving editing. Its core module, MeloDRP, predicts fixed-budget duration ratios, enabling explicit span-wise duration control. For melody-aware duration allocation, MeloDRP fuses phonetic cues with pseudo-MIDI melodic context through cross-attention, while temporal-overlap supervision encourages soft phoneme--note correspondences. We further use a flow-matching mel decoder for audio infilling to synthesize edited regions while preserving surrounding context. In addition, we introduce a duration-aware edited-lyric generation pipeline using WhisperX and an LLM to construct feasible evaluation scenarios. Experiments demonstrate state-of-the-art performance in both objective and subjective evaluations.

Problem

• singing voice editing
• melody preservation
• duration preservation
• text-based editing
• audio infilling

Innovation

• flow-matching
• melody-aware duration control
• audio infilling
• cross-attention
• phoneme-note alignment