Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints

2025-05-26

AI Summary

Existing Neural ODEs struggle to strictly enforce strong algebraic constraints—such as conservation laws—in physical systems, suffering from poor scalability and numerical instability. To address this, we propose Projected Neural ODEs (PNODEs), a novel framework grounded in semi-explicit differential-algebraic equations (DAEs). PNODEs are the first Neural ODE architecture to perform explicit, exact projection onto the constraint manifold at every integration step, thereby guaranteeing strict satisfaction of conservation laws from the outset. Our method unifies and generalizes existing relaxation-based approaches, integrating both iterative and single-step Jacobian decomposition approximations with stability-driven numerical design. Evaluated on six highly constrained benchmark tasks, PNODEs achieve constraint violation errors as low as 10⁻¹⁰, significantly outperforming state-of-the-art baselines in both accuracy and computational efficiency.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Optimization
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Despite the promise of scientific machine learning (SciML) in combining data-driven techniques with mechanistic modeling, existing approaches for incorporating hard constraints in neural differential equations (NDEs) face significant limitations. Scalability issues and poor numerical properties prevent these neural models from being used for modeling physical systems with complicated conservation laws. We propose Manifold-Projected Neural ODEs (PNODEs), a method that explicitly enforces algebraic constraints by projecting each ODE step onto the constraint manifold. This framework arises naturally from semi-explicit differential-algebraic equations (DAEs), and includes both a robust iterative variant and a fast approximation requiring a single Jacobian factorization. We further demonstrate that prior works on relaxation methods are special cases of our approach. PNODEs consistently outperform baselines across six benchmark problems achieving a mean constraint violation error below $10^{-10}$. Additionally, PNODEs consistently achieve lower runtime compared to other methods for a given level of error tolerance. These results show that constraint projection offers a simple strategy for learning physically consistent long-horizon dynamics.

Problem

• Enforcing hard constraints in neural differential equations
• Modeling physical systems with complex conservation laws
• Improving scalability and numerical properties of neural models

Innovation

• Enforces algebraic constraints via manifold projection
• Uses semi-explicit DAEs for robust learning
• Combines iterative and fast approximation methods