Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration

2026-05-13

AI Summary

Image restoration is inherently limited by ill-posed inverse problems, and existing methods lack a systematic theoretical understanding of the relationship between data symmetry and model equivariance, particularly struggling to model real-world data that exhibits only approximate symmetry. This work addresses this gap by introducing, from an optimization perspective, the first quantifiable definition of approximate symmetry at the dataset level. It establishes rigorous bounds linking equivariance error to both data symmetry error and grid resolution, revealing that aligning these factors enables an improved bias-variance trade-off. Building on this insight, the authors propose Sample-Adaptive Equivariant Convolution (SA-Conv), which employs a hypernetwork and learnable transformations to dynamically match the intrinsic symmetry of each input sample. Extensive experiments on super-resolution, denoising, and deraining tasks demonstrate significant performance gains over standard baselines and conventional equivariant models, validating both the theoretical framework and the proposed method.

Abstract

Image restoration is an inherently ill posed inverse problem. Equivariant networks that embed geometric symmetry priors can mitigate this ill posedness and improve performance. However, current understanding of the relationship between network equivariance and data symmetry remains largely heuristic. Particularly for real world data with imperfect symmetry, existing research lacks a systematic theoretical framework to quantify symmetry, select transformation groups, or evaluate model data alignment. To bridge this gap, we conduct an analysis from an optimization perspective and formalize the intrinsic relationship among data symmetry priors, model equivariance, and generalization capability. Specifically, we propose for the first time a quantifiable definition of non strict symmetry at the dataset level (rather than sample level) and use it as a constraint to formulate the restoration inverse problem. We then show that the equivariance for restoration models can be naturally derived from this inverse problems incorporated the proposed symmetry constraints, and that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size. Furthermore, by analyzing the network's empirical risk, we demonstrate that aligning equivariance with data symmetry optimizes the bias variance trade off, minimizing the total expected risk. Guided by these insights, we propose a Sample Adaptive Equivariant Network that uses a hypernetwork and transformation learnable equivariant convolutions to dynamically align with each sample's inherent symmetry. Extensive experiments on super resolution, denoising, and deraining validate our theoretical findings and show significant superiority over standard baselines and traditional equivariant models. Our code and supplementary material are available at https://github.com/tanfy929/SA-Conv.

Problem

• equivariance
• data symmetry
• image restoration
• inverse problem
• symmetry prior

Innovation

• equivariance
• data symmetry
• image restoration
• adaptive networks
• theoretical framework