Separation Capacity of Scattering Networks

2026-06-29

AI Summary

This study addresses the problem of quantifying the feature separation capacity of convolutional neural networks in classification tasks. Building upon Cover’s function counting theory and integrating tools from combinatorics with an analysis of scattering network architectures, we extend the classical theoretical framework to derive, for the first time, a practical expression for the separation capacity specific to scattering networks. Our analysis elucidates the critical influence of architectural components—such as the number of filters and network depth—on separation capability. Furthermore, this work establishes the first theoretically grounded design principles for scattering networks, offering both significant theoretical insight and practical utility for network architecture development.

Abstract

In this paper, we attempt to enhance the theoretical understanding of convolutional neural networks (CNNs) as feature extractors in classification tasks by analyzing them through the lens of Cover's function-counting theory. Specifically, our focus lies on the notion of separation capacity, a combinatorial quantity derived from counting the number of realizable dichotomies (i.e., binary label assignments). Our contributions are threefold. First, we extend Cover's framework by establishing a conceptually insightful and practically useful formulation for the separation capacity. Second, leveraging this formulation, we identify the factors governing the separation capacity of feature extractors that employ a specific CNN architecture, so-called scattering networks, in terms of their network building blocks. Third, we provide practical insights for scattering network design.

Problem

• separation capacity
• scattering networks
• convolutional neural networks
• Cover's theory
• feature extraction

Innovation

• separation capacity
• scattering networks
• Cover's theory
• feature extraction
• dichotomy counting