A Nonlinear Separation Principle: Applications to Neural Networks, Control and Learning

2026-04-16

AI Summary

This work addresses the challenges of stability and output tracking in recurrent neural networks (RNNs) for nonlinear control and implicit deep learning by introducing a nonlinear separation principle that decouples contractive state feedback from observer design, thereby guaranteeing global exponential stability of the closed-loop system. The authors characterize, via linear matrix inequalities (LMIs), the maximal stable weight space of continuous-time RNNs with monotone activation functions and propose an unconstrained algebraic parameterization to construct highly expressive implicit neural networks. By integrating low-gain integral control with graph neural network techniques, the approach achieves parameter-efficient and competitive performance on standard image classification benchmarks while providing rigorous guarantees of stability and accurate reference tracking for RNNs.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper investigates continuous-time and discrete-time firing-rate and Hopfield recurrent neural networks (RNNs), with applications in nonlinear control design and implicit deep learning. First, we introduce a nonlinear separation principle that guarantees global exponential stability for the interconnection of a contracting state-feedback controller and a contracting observer, alongside parametric extensions for robustness and equilibrium tracking. Second, we derive sharp linear matrix inequality (LMI) conditions that guarantee the contractivity of both firing rate and Hopfield neural network architectures. We establish structural relationships among these certificates-demonstrating that continuous-time models with monotone non-decreasing activations maximize the admissible weight space, and extend these stability guarantees to interconnected systems and Graph RNNs. Third, we combine our separation principle and LMI framework to solve the output reference tracking problem for RNN-modeled plants. We provide LMI synthesis methods for feedback controllers and observers, and rigorously design a low-gain integral controller to eliminate steady-state error. Finally, we derive an exact, unconstrained algebraic parameterization of our contraction LMIs to design highly expressive implicit neural networks, achieving competitive accuracy and parameter efficiency on standard image classification benchmarks.

Problem

• contractivity
• recurrent neural networks
• nonlinear control
• stability
• implicit deep learning

Innovation

• nonlinear separation principle
• contracting systems
• linear matrix inequalities (LMIs)
• recurrent neural networks
• implicit deep learning