Distinguishing AI-Generated Music from Edited Audio as a Hard-Negative Robustness Task

2026-08-14

AI Summary

This study addresses the challenge of misclassification in AI music detection caused by edited audio serving as hard negative samples. We propose a robust detection framework that explicitly models edited audio as hard negatives and employs a pretrained PaSST architecture to process raw waveforms. To prevent information leakage, an anchor-song-based data partitioning strategy is designed, while Grad-CAM is utilized to reveal detectable spectral fingerprint features. Experimental results demonstrate that the model achieves a video-level balanced accuracy of 0.811 and an F1 score of 0.836 for AI-generated segments. These findings confirm the framework’s effectiveness in distinguishing between AI-generated and edited audio, validating the critical role of spectral cues in achieving robust detection performance.

Abstract

AI-generated music detectors are commonly evaluated against original songs, but real-world uploads are often remixed, re-encoded, pitch-shifted, or otherwise edited. These edited versions form a difficult negative class: they are not generated by AI, yet they may introduce spectral artifacts that resemble synthetic audio fingerprints. We study this problem as a hard-negative robustness setting for AI-generated music detection, focusing on AI-generated and edited variants derived from the same anchor songs. We compile a YouTube-based dataset of AI, edited, and original variants, using the original tracks only as references, and train a binary AI versus edited detector. Audio is processed as 10-second clips and passed as raw waveforms to a pretrained PaSST spectrogram transformer. To reduce leakage, all splits are performed by anchor song. On the held-out test set, the final video-level system achieves 0.811 balanced accuracy. At clip level, AI-generated clips reach an F1-score of 0.836, while edited clips reach a lower F1-score of 0.720. The results suggest that AI-generated music retains detectable fingerprint-like spectral cues beyond ordinary editing, but the lower edited-class F1-score shows that these cues can still overlap with artifacts from edited audio. Grad-CAM visualizations are used to inspect whether high-confidence predictions rely on localized time-frequency regions.

Problem

• AI-generated music detection
• hard-negative robustness
• edited audio
• spectral artifacts

Innovation

• Hard-Negative Robustness
• PaSST Spectrogram Transformer
• Edited Audio Artifacts
• Anchor Song Splitting
• Raw Waveform Processing