Song Form-aware Full-Song Text-to-Lyrics Generation with Multi-Level Granularity Syllable Count Control

2024-11-20

arXiv.org

AI Summary

Lyric generation faces dual challenges: achieving syllable-level precision while jointly modeling song structure (e.g., verse/chorus). Conventional line-by-line generation often yields semantic discontinuity and prosodic misalignment. This paper proposes the first end-to-end full-song lyric generation framework that unifies song-form awareness with multi-granularity syllabic constraints—operating at the word, phrase, line, and section levels. Methodologically, we design a hierarchical conditional sequence decoder incorporating structure-aware positional encoding, cross-granularity syllable alignment loss, and form-guided attention. Evaluated on a multi-style dataset, our approach achieves an 18.7% improvement in syllable accuracy and a 22.3% gain in structural consistency (F1). Human evaluation by professional lyricists confirms significant gains in naturalness and singability of generated lyrics.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Lyrics generation presents unique challenges, particularly in achieving precise syllable control while adhering to song form structures such as verses and choruses. Conventional line-by-line approaches often lead to unnatural phrasing, underscoring the need for more granular syllable management. We propose a framework for lyrics generation that enables multi-level syllable control at the word, phrase, line, and paragraph levels, aware of song form. Our approach generates complete lyrics conditioned on input text and song form, ensuring alignment with specified syllable constraints. Generated lyrics samples are available at: https://tinyurl.com/lyrics9999

Problem

• Achieve precise syllable control in lyrics generation
• Adhere to song form structures like verses and choruses
• Enable multi-level syllable control at various granularities

Innovation

• Song form-aware lyrics generation framework
• Multi-level syllable count control
• Full-song generation with text input