Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations

2026-07-09

AI Summary

This study addresses the problem of synthesizing guitar audio with target effects while preserving the original musical content, given clean electric guitar recordings and effect labels. To this end, the authors construct a paired dataset of clean and effected audio and propose a unified spectrogram transformation framework conditioned on effect labels to enable controllable effect rendering. The approach integrates a variational autoencoder with a U-Net architecture, modeling both magnitude and log-magnitude spectrograms, and employs sensitivity diagnostics to verify that the model responds meaningfully to effect labels rather than collapsing to a trivial mapping. Experimental results demonstrate that the U-Net variant outperforms the variational autoencoder, particularly for distortion-type effects, and exhibits strong generalization to real-world guitar performances outside the training set.

Abstract

We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio. Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content. Training and evaluation pairs are constructed from EGFxSet real, single tone recordings by assembling matched clean/effected chords, melodies, and mixed timelines. This allows for controlled comparison across effects. We evaluate four neural approaches under a common spectrogram-based transformation setting: two variational autoencoders and two U-Net models that differ in whether they operate on linear or log-magnitude representations. Performance is measured using linear-magnitude spectrogram MSE and Fréchet Audio Distance. The U-Net models outperform the variational autoencoder variants. Per-effect results show that distortion effects are most readily improved, whereas delay and reverb effects exhibit weaker FAD gains despite substantial spectral-error reductions. A conditioning-sensitivity diagnostic provides evidence that the best model responds to target labels rather than collapsing to a single transformation. Our demo website compares two models applied on real-world guitar performances outside training and validation data, providing audio and spectrogram examples of the practical clean-to-effect behavior.

Problem

• clean-to-effect transformation
• guitar audio
• label-conditioned modeling
• audio effects synthesis
• musical content preservation

Innovation

• label-conditioned audio synthesis
• clean-to-effect transformation
• guitar effects modeling
• U-Net audio processing
• spectrogram-based neural audio