SONICS: Synthetic Or Not - Identifying Counterfeit Songs

2024-08-26

arXiv.org

AI Summary

To address the lack of detection methods for end-to-end AI-generated songs—fully synthesized with vocals, accompaniment, lyrics, and stylistic attributes—this paper introduces SONICS, the first dedicated benchmark dataset comprising 97k samples (4,751 hours), filling critical gaps in duration, diversity, and open availability of synthetic music. We formally define the task of AI-generated song detection and propose SpecTTTra, an efficient long-sequence modeling architecture that integrates spectrogram-based representations, a lightweight temporal Transformer, and chunked sequence processing, enhanced by contrastive learning for improved discriminability. Experiments demonstrate that SpecTTTra achieves an 8% higher F1 score than ViT on long-song detection, with 38% faster inference and 26% lower memory consumption; against ConvNeXt, it attains a 1% F1 gain, 20% speedup, and 67% memory reduction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated. Additionally, existing datasets lack music-lyrics diversity, long-duration songs, and open-access fake songs. To address these gaps, we introduce SONICS, a novel dataset for end-to-end Synthetic Song Detection (SSD), comprising over 97k songs (4,751 hours) with over 49k synthetic songs from popular platforms like Suno and Udio. Furthermore, we highlight the importance of modeling long-range temporal dependencies in songs for effective authenticity detection, an aspect entirely overlooked in existing methods. To utilize long-range patterns, we introduce SpecTTTra, a novel architecture that significantly improves time and memory efficiency over conventional CNN and Transformer-based models. For long songs, our top-performing variant outperforms ViT by 8% in F1 score, is 38% faster, and uses 26% less memory, while also surpassing ConvNeXt with a 1% F1 score gain, 20% speed boost, and 67% memory reduction.

Problem

• Detecting end-to-end AI-generated songs
• Addressing lack of diverse and long-duration song datasets
• Improving time and memory efficiency in song authenticity detection

Innovation

• SONICS dataset for synthetic detection
• SpecTTTra architecture improves efficiency
• Focuses on long-range temporal dependencies