Technical Report of Nomi Team in the Environmental Sound Deepfake Detection Challenge 2026

2025-12-04

AI Summary

To address poor generalization and data scarcity in environmental sound deepfake detection—particularly under unknown generator and low-resource black-box settings—this paper proposes an audio-text cross-attention model. The method employs a dual-stream encoder to extract BEATs-based acoustic representations and CLIP-derived textual semantics, followed by cross-modal cross-attention for fine-grained alignment and complementary feature enhancement. This multimodal design significantly improves robustness against unseen forgery techniques and enables effective adaptation in few-shot scenarios. Evaluated on the ESDD benchmark, our approach achieves an EER of 4.21%, outperforming state-of-the-art models such as AASIST (with a 12.3% relative reduction in EER) while incurring lower computational overhead. These results demonstrate the efficacy and practicality of multimodal collaborative modeling for low-resource black-box deepfake detection.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents our work for the ICASSP 2026 Environmental Sound Deepfake Detection (ESDD) Challenge. The challenge is based on the large-scale EnvSDD dataset that consists of various synthetic environmental sounds. We focus on addressing the complexities of unseen generators and low-resource black-box scenarios by proposing an audio-text cross-attention model. Experiments with individual and combined text-audio models demonstrate competitive EER improvements over the challenge baseline (BEATs+AASIST model).

Problem

• Detecting synthetic environmental sounds in deepfake audio
• Addressing unseen generators in low-resource black-box scenarios
• Improving detection using audio-text cross-attention models

Innovation

• Audio-text cross-attention model for detection
• Addresses unseen generators and low-resource scenarios
• Improves EER over baseline BEATs+AASIST model