BEAT2AASIST model with layer fusion for ESDD 2026 Challenge

2025-12-17

AI Summary

To address the challenges of increasingly realistic environmental sound deepfakes and insufficient generalization in ESDD detection, this paper proposes an end-to-end dual-branch BEATs-AASIST framework. The method innovatively introduces a frequency-domain and channel-wise feature splitting mechanism, incorporates a top-k Transformer layer fusion strategy—with three variants: concatenation, CNN-gated fusion, and SE-gated fusion—and integrates vocoder-driven data augmentation to enhance robustness against unseen forgery methods. Evaluated on the official ESDD 2026 Challenge test set, the proposed approach achieves state-of-the-art or highly competitive performance across all tracks, significantly outperforming baseline models. These results demonstrate its strong cross-type generalization capability—spanning diverse forgery categories and generative models—while maintaining high detection accuracy and robustness.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recent advances in audio generation have increased the risk of realistic environmental sound manipulation, motivating the ESDD 2026 Challenge as the first large-scale benchmark for Environmental Sound Deepfake Detection (ESDD). We propose BEAT2AASIST which extends BEATs-AASIST by splitting BEATs-derived representations along frequency or channel dimension and processing them with dual AASIST branches. To enrich feature representations, we incorporate top-k transformer layer fusion using concatenation, CNN-gated, and SE-gated strategies. In addition, vocoder-based data augmentation is applied to improve robustness against unseen spoofing methods. Experimental results on the official test sets demonstrate that the proposed approach achieves competitive performance across the challenge tracks.

Problem

• Detects environmental sound deepfakes to counter audio manipulation risks
• Enhances feature representation with layer fusion and dual-branch processing
• Improves robustness against unseen spoofing via data augmentation

Innovation

• BEAT2AASIST model with dual AASIST branches for feature processing
• Top-k transformer layer fusion with gated strategies for enriched features
• Vocoder-based data augmentation to enhance robustness against spoofing