Classic Video Denoising in a Machine Learning World: Robust, Fast, and Controllable

2025-04-04

AI Summary

Deep video denoising methods suffer from poor robustness, slow inference, and limited controllability, while traditional approaches rely heavily on manual parameter tuning—creating a dual bottleneck for real-world video denoising. Method: We propose the first differentiable traditional video denoising framework, fully differentiating the classical non-local means and optical-flow-guided pipeline, and integrating a lightweight CNN-based parameter prediction network for neural-driven, end-to-end automatic parameter optimization. Contribution/Results: Our method preserves the high robustness and real-time performance of traditional algorithms while enabling fine-grained user-controllable adjustments. Experiments on real-noise videos demonstrate significantly superior robustness over state-of-the-art deep methods, 3–5× faster inference speed, and complete elimination of manual parameter tuning—achieving, for the first time, an organic integration of classical algorithmic strengths with deep learning advantages.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Denoising is a crucial step in many video processing pipelines such as in interactive editing, where high quality, speed, and user control are essential. While recent approaches achieve significant improvements in denoising quality by leveraging deep learning, they are prone to unexpected failures due to discrepancies between training data distributions and the wide variety of noise patterns found in real-world videos. These methods also tend to be slow and lack user control. In contrast, traditional denoising methods perform reliably on in-the-wild videos and run relatively quickly on modern hardware. However, they require manually tuning parameters for each input video, which is not only tedious but also requires skill. We bridge the gap between these two paradigms by proposing a differentiable denoising pipeline based on traditional methods. A neural network is then trained to predict the optimal denoising parameters for each specific input, resulting in a robust and efficient approach that also supports user control.

Problem

• Bridging gap between deep learning and traditional video denoising methods
• Achieving robust denoising for diverse real-world noise patterns
• Enabling fast controllable denoising without manual parameter tuning

Innovation

• Differentiable pipeline based on traditional methods
• Neural network predicts optimal denoising parameters
• Robust, efficient, and supports user control