Axiomatizing Neural Networks via Pursuit of Subspaces

2026-05-19

AI Summary

Although deep neural networks exhibit remarkable performance, their black-box nature hinders systematic understanding and principled design. To address this, this work proposes the "Pursuit of Subspaces" (PoS) hypothesis, introducing an axiomatic approach into deep learning theory for the first time. The authors develop a unified framework grounded in geometric axioms to characterize representational structures, computational mechanisms, and generalization behavior in both shallow and deep networks. By integrating subspace analysis, representation learning, and generalization theory, the framework establishes a formal, interpretable geometric foundation for neural networks. This advancement paves the way toward a predictive and theoretically rigorous understanding of deep learning systems.

Abstract

While deep neural networks have achieved remarkable success across a wide range of domains, their underlying mechanisms remain poorly understood, and they are often regarded as black boxes. This gap between empirical performance and theoretical understanding poses a challenge analogous to the pre-axiomatic stage of classical geometry. In this work, we introduce the Pursuit of Subspaces (PoS) hypothesis, an axiomatic framework that formulates neural network behavior through a set of geometric postulates. These axioms, together with their derived consequences, provide a unified perspective on representation, computation, and generalization in both shallow and deep architectures. We show that this framework yields geometric explanations for fundamental questions in deep learning, including representation structure, architectural mechanisms, and generalization behavior, offering a principled step toward a coherent theoretical foundation.

Problem

• neural networks
• theoretical understanding
• black box
• axiomatization
• generalization

Innovation

• axiomatic framework
• subspace pursuit
• geometric postulates
• neural network theory
• representation geometry