From Self-Supervised Speech Models to Mixture-of-Experts for Robust Anti-Spoofing

2026-06-12

AI Summary

This work addresses the limited generalization of current anti-spoofing systems when confronted with speech generated by unseen synthesis methods. To enhance robustness, the study introduces a Mixture-of-Experts (MoE) architecture into self-supervised speech representation models for the first time. Specifically, the standard feed-forward modules in key encoding layers are replaced with multiple expert networks, and a learnable per-layer gating mechanism is designed to enable collaborative modeling of complementary acoustic features while preserving the pre-trained representational capabilities. The proposed approach significantly improves generalization to previously unseen spoofed speech, reducing the macro-averaged equal error rate (EER) from 5.46% to 4.81% across 14 spoofing datasets—a relative improvement of 11.9%.

Abstract

Recent advances in speech generation have significantly improved the naturalness of synthetic speech, making spoofing detection increasingly challenging. A key limitation of current anti-spoofing systems is their limited robustness to unseen synthesis methods. In this work, we transform a self-supervised speech representation model into a Mixture-of-Experts (MoE) architecture to improve generalization. Feed-forward blocks in selected encoder layers are replaced by multiple expert networks controlled by a layer-wise gating mechanism, allowing experts to capture complementary acoustic patterns while preserving the representations learned during self-supervised pretraining. We further analyze the architectural choices affecting the performance of this MoE conversion and investigate the activation behavior of the experts. The proposed approach is evaluated on 14 spoofing datasets and reduces the macro EER from 5.46% to 4.81%, corresponding to 11.9% relative improvement over the baseline.

Problem

• anti-spoofing
• speech synthesis
• robustness
• unseen spoofing methods
• synthetic speech

Innovation

• Mixture-of-Experts
• self-supervised speech representation
• anti-spoofing
• robust generalization
• expert gating