CodePDE: An Inference Framework for LLM-driven PDE Solver Generation

2025-05-13

AI Summary

Traditional PDE solvers rely heavily on expert knowledge and incur high computational costs, while neural-network-based solvers require large labeled datasets and lack interpretability. To address these limitations, this paper introduces the first pure reasoning-time large language model (LLM) framework dedicated to PDE solving. Our approach formulates PDE solving as a code-generation task—requiring no fine-tuning—and leverages reasoning-time search, symbolic-numerical co-verification, self-debugging, self-refinement, and test-time scaling to autonomously generate executable, verifiable numerical solvers. Evaluated on diverse canonical PDEs, our method significantly outperforms conventional solvers. We further uncover intrinsic LLM behaviors governing accuracy, efficiency, and numerical scheme selection. The complete codebase is open-sourced, establishing a new paradigm for interpretable, low-data-dependency AI in scientific computing.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Partial differential equations (PDEs) are fundamental to modeling physical systems, yet solving them remains a complex challenge. Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive, while neural-network-based solvers require large training datasets and often lack interpretability. In this work, we frame PDE solving as a code generation task and introduce CodePDE, the first inference framework for generating PDE solvers using large language models (LLMs). Leveraging advanced inference-time algorithms and scaling strategies, CodePDE unlocks critical capacities of LLM for PDE solving: reasoning, debugging, selfrefinement, and test-time scaling -- all without task-specific tuning. CodePDE achieves superhuman performance across a range of representative PDE problems. We also present a systematic empirical analysis of LLM generated solvers, analyzing their accuracy, efficiency, and numerical scheme choices. Our findings highlight the promise and the current limitations of LLMs in PDE solving, offering a new perspective on solver design and opportunities for future model development. Our code is available at https://github.com/LithiumDA/CodePDE.

Problem

• Generates PDE solvers using LLMs without task-specific tuning
• Addresses complexity and cost of traditional numerical PDE solvers
• Overcomes data needs and interpretability issues of neural-network solvers

Innovation

• LLM-driven PDE solver generation framework
• Advanced inference-time algorithms and scaling
• Superhuman performance without task-specific tuning