DUE: A Deep Learning Framework and Library for Modeling Unknown Equations

2025-04-14

AI Summary

This work addresses the fundamental scientific challenge of automatically discovering unknown dynamical equations directly from measurement or simulation data. We propose DUE—the first unified, differentiable deep learning framework capable of end-to-end modeling of diverse differential equations, including ODEs, PDEs, DAEs, and SDEs. Methodologically, we introduce a novel hybrid architecture integrating Operator Semigroup Networks (OSG-Nets) with Transformers, overcoming key bottlenecks in non-autonomous dynamics, partial observability, and high-dimensional dimensionality reduction; we further incorporate automatic differentiation, neural operator learning, and surrogate modeling. The open-source DUE library combines theoretical rigor with engineering practicality, achieving high-accuracy equation discovery and efficient surrogate modeling across multiple complex systems. It significantly reduces computational overhead compared to traditional numerical solvers, establishing a new paradigm for data-driven dynamical system modeling.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Equations, particularly differential equations, are fundamental for understanding natural phenomena and predicting complex dynamics across various scientific and engineering disciplines. However, the governing equations for many complex systems remain unknown due to intricate underlying mechanisms. Recent advancements in machine learning and data science offer a new paradigm for modeling unknown equations from measurement or simulation data. This paradigm shift, known as data-driven discovery or modeling, stands at the forefront of AI for science, with significant progress made in recent years. In this paper, we introduce a systematic framework for data-driven modeling of unknown equations using deep learning. This versatile framework is capable of learning unknown ODEs, PDEs, DAEs, IDEs, SDEs, reduced or partially observed systems, and non-autonomous differential equations. Based on this framework, we have developed Deep Unknown Equations (DUE), an open-source software package designed to facilitate the data-driven modeling of unknown equations using modern deep learning techniques. DUE serves as an educational tool for classroom instruction, enabling students and newcomers to gain hands-on experience with differential equations, data-driven modeling, and contemporary deep learning approaches such as FNN, ResNet, generalized ResNet, operator semigroup networks (OSG-Net), and Transformers. Additionally, DUE is a versatile and accessible toolkit for researchers across various scientific and engineering fields. It is applicable not only for learning unknown equations from data but also for surrogate modeling of known, yet complex, equations that are costly to solve using traditional numerical methods. We provide detailed descriptions of DUE and demonstrate its capabilities through diverse examples, which serve as templates that can be easily adapted for other applications.

Problem

• Modeling unknown differential equations from data
• Learning diverse equation types with deep learning
• Providing an open-source toolkit for researchers and educators

Innovation

• Deep learning framework for unknown equations modeling
• Open-source library with modern deep learning techniques
• Versatile toolkit for ODEs, PDEs, and surrogate modeling