SongEcho: Towards Cover Song Generation via Instance-Adaptive Element-wise Linear Modulation

2026-02-23

AI Summary

Existing approaches struggle to simultaneously preserve melodic fidelity and convey expressive emotion in cover song generation. This work formulates the task as a conditional generation problem, leveraging the original vocal melody and textual prompts as conditions to jointly synthesize a new vocal performance and its accompanying instrumentation, thereby enabling controllable and expressive reinterpretation. To this end, the authors propose an Instance-Adaptive Element-wise Linear Modulation (IA-EiLM) mechanism, which integrates enhanced temporal alignment via EiLM modulation with an Instance-Adaptive Conditional Refinement (IACR) module for latent state interaction. Additionally, they introduce Suno70k, a high-quality AI-generated song dataset. Experimental results demonstrate that the proposed method outperforms current state-of-the-art models in generation quality across multiple datasets while reducing trainable parameters to less than 30%, significantly improving both efficiency and performance.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Cover songs constitute a vital aspect of musical culture, preserving the core melody of an original composition while reinterpreting it to infuse novel emotional depth and thematic emphasis. Although prior research has explored the reinterpretation of instrumental music through melody-conditioned text-to-music models, the task of cover song generation remains largely unaddressed. In this work, we reformulate our cover song generation as a conditional generation, which simultaneously generates new vocals and accompaniment conditioned on the original vocal melody and text prompts. To this end, we present SongEcho, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation. To enhance the conditioning injection mechanism, we extend Feature-wise Linear Modulation (FiLM) to an Element-wise Linear Modulation (EiLM), to facilitate precise temporal alignment in melody control. For conditional representations, we propose Instance-Adaptive Condition Refinement (IACR), which refines conditioning features by interacting with the hidden states of the generative model, yielding instance-adaptive conditioning. Additionally, to address the scarcity of large-scale, open-source full-song datasets, we construct Suno70k, a high-quality AI song dataset enriched with comprehensive annotations. Experimental results across multiple datasets demonstrate that our approach generates superior cover songs compared to existing methods, while requiring fewer than 30% of the trainable parameters. The code, dataset, and demos are available at https://github.com/lsfhuihuiff/SongEcho_ICLR2026.

Problem

• cover song generation
• conditional generation
• melody conditioning
• text-to-music
• vocal and accompaniment synthesis

Innovation

• Instance-Adaptive Element-wise Linear Modulation
• Cover Song Generation
• Conditional Music Generation
• Feature-wise Linear Modulation
• AI Music Dataset