Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising

2026-05-05

AI Summary

This work addresses the insufficient out-of-distribution robustness of image-to-image prediction models under global contrast and brightness shifts by proposing a parameter-free Normalization Equivariance Wrapper (WNE). WNE formulates normalization equivariance (NE) as an input–output parameterization problem and, for the first time, fully characterizes the class of NE functions. Leveraging a normalize-process-denormalize decomposition framework, it transforms NE from an internal architectural constraint into a universal, plug-and-play external mechanism compatible with any backbone—including Transformers—without requiring architectural modifications or incurring inference overhead. Experiments demonstrate that WNE substantially enhances the robustness of both CNNs and Transformers in single-noise-mismatch blind denoising tasks, achieving this improvement with zero GPU runtime latency, whereas conventional NE approaches introduce up to a 1.6× slowdown.

Abstract

Normalization Equivariance (NE), equivariance to global contrast and brightness transforms, improves robustness to distribution shift in image-to-image prediction. Existing methods enforce this prior by constraining internal layers to NE-compatible families, limiting compatibility with standard components such as attention and LayerNorm, and adding runtime cost. We characterize the full NE function class: a function is NE if and only if it admits a normalize-process-denormalize factorization. This turns exact NE enforcement, for the ideal wrapper, from an internal architectural constraint into an input-output parameterization problem, allowing a parameter-free wrapper (WNE) to enforce NE around any backbone, including transformers. In a single-noise mismatch diagnostic for blind denoising, the wrapper improves CNN and transformer robustness with no measurable GPU overhead; architectural NE baselines incur up to a 1.6x slowdown.

Problem

• Normalization Equivariance
• distribution shift
• image denoising
• architectural constraint
• robustness

Innovation

• Normalization Equivariance
• normalize-process-denormalize factorization
• parameter-free wrapper
• distribution shift robustness
• blind image denoising