Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning

2025-10-13

AI Summary

To address the weak generalization capability of deep neural networks and their susceptibility to domain shifts, this paper proposes Deep Edge Filter (DEF), the first differentiable high-pass filtering layer embedded within deep learning frameworks to explicitly disentangle high-frequency components (task-relevant semantics) from low-frequency ones (domain-specific biases). DEF extracts high-frequency features via low-pass filtered reconstruction followed by subtraction, and validates their generalization capacity through sparsity analysis. The method demonstrates cross-modal (vision, text, 3D, audio) and cross-architecture (CNN, Transformer, etc.) compatibility, achieving significant improvements in generalization and robustness across multiple benchmarks. Key contributions include: (1) revealing a strong empirical correlation between high-frequency feature representations and model generalization; (2) introducing the first plug-and-play, generalization-oriented differentiable filtering module; and (3) establishing an interpretable, scalable paradigm for bias disentanglement.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by our hypothesis that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components of deep features. By subtracting low-pass filtered outputs from original features, our approach isolates generalizable representations while preserving architectural integrity. Experimental results across diverse domains such as Vision, Text, 3D, and Audio demonstrate consistent performance improvements regardless of model architecture and data modality. Analysis reveals that our method induces feature sparsification and effectively isolates high-frequency components, providing empirical validation of our core hypothesis. The code is available at https://github.com/dongkwani/DeepEdgeFilter.

Problem

• Improves model generalizability by filtering deep neural network features
• Isolates semantic information from domain biases in frequency components
• Enhances performance across vision, text, 3D and audio domains

Innovation

• High-pass filtering applied to deep features
• Isolates generalizable high-frequency components
• Preserves architectural integrity across domains