PDE-Controller: LLMs for Autoformalization and Reasoning of PDEs

2025-02-03

AI Summary

This work addresses the challenge of enabling large language models (LLMs) to reliably understand physical semantics, formalize natural-language control instructions, and generate robust control strategies for partial differential equation (PDE)-governed systems—a longstanding limitation in scientific LLM applications. We propose the first LLM framework specifically designed for PDE control, featuring a million-scale, physics-consistent synthetic dataset and a dedicated evaluation benchmark. Our method integrates instruction tuning, mathematical reasoning modeling, program synthesis, and multi-stage chain-of-thought reasoning, synergistically leveraging both human-annotated and high-fidelity synthetic data. Experiments demonstrate substantial improvements over state-of-the-art open-source and GPT-series models across PDE instruction formalization, physical reasoning, and control program synthesis. Notably, the generated controllers achieve up to 62% higher real-world control efficacy, establishing deep coupling between linguistic understanding and first-principles physical system modeling.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

While recent AI-for-math has made strides in pure mathematics, areas of applied mathematics, particularly PDEs, remain underexplored despite their significant real-world applications. We present PDE-Controller, a framework that enables large language models (LLMs) to control systems governed by partial differential equations (PDEs). Our approach enables LLMs to transform informal natural language instructions into formal specifications, and then execute reasoning and planning steps to improve the utility of PDE control. We build a holistic solution comprising datasets (both human-written cases and 2 million synthetic samples), math-reasoning models, and novel evaluation metrics, all of which require significant effort. Our PDE-Controller significantly outperforms prompting the latest open-source and GPT models in reasoning, autoformalization, and program synthesis, achieving up to a 62% improvement in utility gain for PDE control. By bridging the gap between language generation and PDE systems, we demonstrate the potential of LLMs in addressing complex scientific and engineering challenges. We will release all data, model checkpoints, and code at https://pde-controller.github.io/.

Problem

• Large Language Models
• Partial Differential Equations
• Scientific and Engineering Problems

Innovation

• PDE-Controller
• Large Language Models
• PDE Control Improvement