QC-GAN: A Parameter-Efficient Quaternion Conformer GAN for High-Fidelity Speech Enhancement

2026-06-16

AI Summary

This work addresses the challenge of balancing model efficiency and performance in high-fidelity speech enhancement by proposing QC-GAN, a novel framework that integrates quaternion representations with the Conformer architecture for the first time. By leveraging Hamiltonian products to jointly model magnitude and phase in a structured manner, QC-GAN preserves their intrinsic correlation while substantially reducing parameter count. The approach further incorporates the MetricGAN training strategy and a metric learning-based discriminator to optimize perceptual quality. On the VoiceBank+DEMAND dataset, the model achieves a PESQ score of 3.48 with only 0.89 million parameters, and even a compact 35K-parameter variant attains 3.23—significantly outperforming conventional methods. Strong generalization capability is also demonstrated on the DNS-Challenge 3 benchmark.

Abstract

We propose a parameter-efficient speech enhancement framework, Quaternion Conformer GAN (QC-GAN), which combines a Quaternion Conformer generator with MetricGAN-based training. The Hamilton product encodes the magnitude and phase via structured weight sharing, reducing the number of layer parameters while preserving their interdependencies. A metric-learning discriminator was employed to maximize perceptual quality by optimizing the approximate perceptual evaluation scores. On the VoiceBank+DEMAND dataset, QC-GAN achieved a Perceptual Evaluation of Speech Quality (PESQ) score of 3.48 with only 0.89M parameters, delivering a performance comparable to state-of-the-art models at less than half their size. A 35K-parameter variant achieved a PESQ score of 3.23, surpassing conventional methods with significantly fewer parameters. Evaluation on the DNS-Challenge 3 dataset further confirmed generalization to real-world conditions.

Problem

• speech enhancement
• parameter efficiency
• high-fidelity
• perceptual quality
• lightweight model

Innovation

• Quaternion Conformer
• Parameter-Efficient GAN
• Hamilton Product
• MetricGAN
• Speech Enhancement