Neural Differential Algebraic Equations

2024-03-19

arXiv.org

AI Summary

This work addresses time-series systems governed by implicit algebraic constraints—such as conservation laws—by proposing the first learnable Neural Differential-Algebraic Equation (Neural DAE) modeling framework. Methodologically, it extends the universal differential equation paradigm to the DAE setting, integrating neural ordinary differential equations, physics-informed embedding, implicit differential solvers, and discrepancy modeling to enable joint mechanism- and data-driven learning. The key contribution lies in enabling concurrent identification of explicit dynamics and implicit constraints, while ensuring robustness to measurement noise and strong extrapolation capability. Experiments on tank-pipe and pump-tank-pipe networks demonstrate successful inverse problem solving, precise decoupling of data-driven trends from underlying physical mechanisms, high-fidelity system modeling, and quantitative error attribution.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Differential-Algebraic Equations (DAEs) describe the temporal evolution of systems that obey both differential and algebraic constraints. Of particular interest are systems that contain implicit relationships between their components, such as conservation relationships. Here, we present Neural Differential-Algebraic Equations (NDAEs) suitable for data-driven modeling of DAEs. This methodology is built upon the concept of the Universal Differential Equation; that is, a model constructed as a system of Neural Ordinary Differential Equations informed by theory from particular science domains. In this work, we show that the proposed NDAEs abstraction is suitable for relevant system-theoretic data-driven modeling tasks. Presented examples include (i) the inverse problem of tank-manifold dynamics and (ii) discrepancy modeling of a network of pumps, tanks, and pipes. Our experiments demonstrate the proposed method's robustness to noise and extrapolation ability to (i) learn the behaviors of the system components and their interaction physics and (ii) disambiguate between data trends and mechanistic relationships contained in the system.

Problem

• Learning unknown components of Differential-Algebraic Equations from data
• Modeling systems with implicit relationships and conservation constraints
• Disambiguating data trends from mechanistic relationships in systems

Innovation

• Operator Splitting for Neural DAEs learning
• Universal Differential Equation framework
• Robust noise handling and extrapolation