Maxout Polytopes

2025-09-25

AI Summary

This work investigates the geometric and combinatorial properties of *maxout polytopes*—the convex polyhedral regions induced by maxout neural networks. Characterizing the piecewise linear structure of maxout networks remains challenging due to their nontrivial activation geometry. Method: Leveraging tools from computational geometry, combinatorial topology, and neural representation theory, we analyze feedforward maxout networks with nonnegative first-layer weights. Contribution/Results: (i) We fully characterize the parameter space structure and extremal *f*-vectors for shallow networks; (ii) we uncover a hierarchical generation mechanism of separating hypersurfaces as network depth increases; (iii) we prove that in generic, bottleneck-free maxout networks, all activation regions are *cubical polytopes*—i.e., all faces are homeomorphic to cubes—a fundamental geometric property not systematically established for ReLU or other piecewise linear networks. These results provide a novel geometric framework for understanding the expressive capacity of deep piecewise linear models.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Social media analysis through the lenses of networks
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Maxout polytopes are defined by feedforward neural networks with maxout activation function and non-negative weights after the first layer. We characterize the parameter spaces and extremal f-vectors of maxout polytopes for shallow networks, and we study the separating hypersurfaces which arise when a layer is added to the network. We also show that maxout polytopes are cubical for generic networks without bottlenecks.

Problem

• Characterizing parameter spaces and f-vectors of maxout polytopes
• Studying separating hypersurfaces when adding network layers
• Showing maxout polytopes are cubical for generic bottleneck-free networks

Innovation

• Maxout polytopes defined by neural networks
• Characterizing parameter spaces and f-vectors
• Showing maxout polytopes are cubical