Function-Counting Theory for Low-Dimensional Data Structures

2026-07-01

AI Summary

This work addresses the limitations of classical function counting theory, which neglects the intrinsic structure of real-world data and thus fails to explain the strong classification performance of models on low-dimensional datasets. For the first time, the authors explicitly incorporate the low-dimensional geometric structure of data into Cover’s function counting framework by revising the general position assumption, thereby constructing a binary classification theory tailored to low-dimensional settings. Building on this foundation, they derive a data-structure-dependent expression for dichotomies and extend analyses of separation capacity and generalization bounds to low-dimensional scenarios. This study establishes a rigorous mathematical basis for understanding how low-dimensional structure influences classification capacity, offering a novel perspective on the empirical success of deep learning on real data.

Abstract

The success of deep learning models in classification and regression is widely attributed to the low-dimensional structure that real-world data tend to exhibit, despite their high-dimensional representation. This work attempts to provide a mathematical framework for binary classification on low-dimensional data, building on Cover's (1965) function-counting theory. With our framework, we aim to address the question of how the low-dimensional structure of the data affects the classification capabilities of learning models. Cover's theory relies on a general position assumption that blinds it to the underlying data structure. We refine this assumption to account for the low-dimensionality of the data and derive dichotomy counts that reflect the data structure. We further extend Cover's separation capacity and problem of generalization to the low-dimensional setting, enabling the impact of the underlying data structure on both to be analyzed.

Problem

• low-dimensional data
• function-counting theory
• classification capacity
• data structure
• Cover's theorem

Innovation

• function-counting theory
• low-dimensional data
• binary classification
• dichotomy count
• generalization