Scale-invariant Gaussian derivative residual networks

2026-03-03

AI Summary

This work addresses the limited generalization of deep networks to unseen image scales during training by proposing GaussDerResNets, a novel architecture that integrates residual connections into Gaussian derivative layers for the first time, yielding a provably strictly scale-invariant deep model. By cascading scale-covariant Gaussian derivative residual blocks that combine scale-covariant convolutions with depthwise separable convolutions, the method achieves strong cross-scale generalization while reducing both parameter count and computational cost. Theoretical analysis provides rigorous proofs of scale covariance and invariance in arbitrary dimensions. Experiments demonstrate that the model attains superior scale generalization and scale selection capabilities on multi-scale variants of STL-10, Fashion-MNIST, and CIFAR-10, maintaining high accuracy with significantly reduced computational overhead.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Non-convex Optimization

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Generalisation across image scales remains a fundamental challenge for deep networks, which often fail to handle images at scales not seen during training (the out-of-distribution problem). In this paper, we present provably scale-invariant Gaussian derivative residual networks (GaussDerResNets), constructed out of scale-covariant Gaussian derivative residual blocks coupled in cascade, aimed at addressing this problem. By adding residual skip connections to the previous notion of Gaussian derivative layers, deeper networks with substantially increased accuracy can be constructed, while preserving very good scale generalisation properties at the higher level of accuracy. Explicit proofs are provided regarding the underlying scale-covariant and scale-invariant properties in arbitrary dimensions. To analyse the ability of GaussDerResNets to generalise to new scales, we apply them on the new rescaled version of the STL-10 dataset, where training is done at a single fixed scale and evaluation is performed on multiple copies of the test set, each rescaled to a single distinct spatial scale, with scale factors extending over a range of 4. We also conduct similar systematic experiments on the rescaled versions of Fashion-MNIST and CIFAR-10 datasets. Experimentally, we demonstrate that the GaussDerResNets have strong scale generalisation and scale selection properties on all the three rescaled datasets. In our ablation studies, we investigate different architectural variants of GaussDerResNets, demonstrating that basing the architecture on depthwise-separable convolutions allows for decreasing both the number of parameters and the amount of computations, with reasonably maintained accuracy and scale generalisation.

Problem

• scale generalisation
• out-of-distribution
• image scale
• deep networks
• scale invariance

Innovation

• scale invariance
• Gaussian derivative
• residual networks
• scale generalisation
• depthwise-separable convolution