ALL-FEM: Agentic Large Language models Fine-tuned for Finite Element Methods

2026-01-08

AI Summary

This work proposes the first autonomous simulation system that integrates an agent-based architecture with domain-finetuned large language models (LLMs) to enable end-to-end modeling and solution of solid mechanics, fluid dynamics, and multiphysics problems. Addressing the limitations of conventional LLMs—which often hallucinate, lack awareness of variational structures, and fail to close the loop from problem description to verified solutions—the system incorporates retrieval-augmented multi-LLM code generation and filtering, finetuned models spanning 3B to 120B parameters, multi-agent collaboration, and runtime feedback mechanisms. A high-quality corpus of over a thousand FEniCS codes was curated to support training and evaluation. On a benchmark suite of 39 nonlinear elasticity, plasticity, and non-Newtonian fluid problems, the GPT OSS 120B model achieved a code generation success rate of 71.79%, substantially outperforming non-agent-based approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Finite element (FE) analysis guides the design and verification of nearly all manufactured objects. It is at the core of computational engineering, enabling simulation of complex physical systems, from fluids and solids to multiphysics systems. However, implementing FE codes and analyzing simulation results demands expertise across numerical analysis, continuum mechanics, and programming. Conventional Large Language Models (LLMs) can generate FE code, but they hallucinate, lack awareness of variational structures, and cannot close the loop from problem statement to a verified solution. Here, we propose ALL-FEM, an autonomous simulation system that integrates agentic AI with domain-specific, fine-tuned LLMs for FEniCS code generation across solid, fluid, and multiphysics applications. We construct a corpus of 1000+ verified FEniCS scripts by combining 500+ curated expert codes with a retrieval-augmented, multi-LLM pipeline that generates and filters codes for diverse PDEs, geometries, and boundary conditions. We used the corpus to fine-tune LLMs with 3B to 120B parameters. Our agentic framework orchestrates specialized agents, powered by fine-tuned LLMs, to formulate problems as PDEs, generate and debug code and visualize the results. We evaluated the system on 39 benchmarks that include problems of linear/nonlinear elasticity, plasticity, Newtonian/non-Newtonian flow, thermofluids, fluid-structure interaction, phase separation, and transport on moving domains. Embedded in a multi-agent workflow with runtime feedback, the best fine-tuned model (GPT OSS 120B) achieves code-level success of 71.79%, outperforming a non-agentic deployment of GPT 5 Thinking. By showing that relatively small, fine-tuned LLMs, orchestrated through agentic frameworks, can automate FE workflows, ALL-FEM offers a blueprint for autonomous simulation systems in computational science and engineering.

Problem

• Finite Element Methods
• Large Language Models
• Code Generation
• Computational Engineering
• PDE Simulation

Innovation

• Agentic AI
• Fine-tuned LLMs
• Finite Element Methods
• FEniCS
• Autonomous Simulation