🤖 AI Summary
This paper investigates the universal function approximation (UAP) capability of controlled dynamical systems, specifically whether the minimal control family—comprising only an affine mapping and a single ReLU nonlinearity—can generate orientation-preserving diffeomorphic flow maps on compact sets.
Method: We model the system via ordinary differential equation (ODE) flows, leverage diffeomorphism approximation theory, and conduct rigorous controllability and realizability analysis.
Contribution/Results: We formally define and prove, for the first time, that this minimal control family achieves UAP; removing the ReLU renders it incapable. We further derive lightweight sufficient conditions—e.g., affine invariance—for UAP and uncover fundamental connections between neural network expressivity and control-theoretic principles. These results provide a rigorous mathematical foundation for architectural design and theoretical validation of flow-based generative models such as Neural ODEs.
📝 Abstract
The universal approximation property (UAP) holds a fundamental position in deep learning, as it provides a theoretical foundation for the expressive power of neural networks. It is widely recognized that a composition of linear and nonlinear functions, such as the rectified linear unit (ReLU) activation function, can approximate continuous functions on compact domains. In this paper, we extend this efficacy to a scenario containing dynamical systems with controls. We prove that the control family $mathcal{F}_1$ containing all affine maps and the nonlinear ReLU map is sufficient for generating flow maps that can approximate orientation-preserving (OP) diffeomorphisms on any compact domain. Since $mathcal{F}_1$ contains only one nonlinear function and the UAP does not hold if we remove the nonlinear function, we call $mathcal{F}_1$ a minimal control family for the UAP. On this basis, several mild sufficient conditions, such as affine invariance, are established for the control family and discussed. Our results reveal an underlying connection between the approximation power of neural networks and control systems and could provide theoretical guidance for examining the approximation power of flow-based models.
