SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Models

2025-05-28

arXiv.org

AI Summary

Current large language models (LLMs) lack sufficient Simulink-domain pretraining data, rendering them unreliable for generating complete, executable Simulink simulation models directly from natural-language requirements. To address this, we propose the first multimodal agent framework specifically designed for Simulink modeling. Our approach integrates graph-structured visual understanding of Simulink diagrams, a domain-specific knowledge base, and a modular role-based collaboration mechanism—featuring specialized agents such as an investigator and a debug locator—to enable interpretable and reproducible end-to-end model generation. Crucially, the framework jointly models the visual representation and symbolic logic of Simulink models, supporting automated generation, debugging, and formal verification of simulation models from textual specifications. Evaluated on representative control and signal processing tasks, our method achieves significant improvements in code generation accuracy and structural completeness, demonstrating both technical efficacy and engineering practicality.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent advances in large language models (LLMs) have shown impressive performance in mathematical reasoning and code generation. However, LLMs still struggle in the simulation domain, particularly in generating Simulink models, which are essential tools in engineering and scientific research. Our preliminary experiments indicate that LLM agents often fail to produce reliable and complete Simulink simulation code from text-only inputs, likely due to the lack of Simulink-specific data in their pretraining. To address this challenge, we propose SimuGen, a multimodal agent-based framework that automatically generates accurate Simulink simulation code by leveraging both the visual Simulink diagram and domain knowledge. SimuGen coordinates several specialized agents, including an investigator, unit test reviewer, code generator, executor, debug locator, and report writer, supported by a domain-specific knowledge base. This collaborative and modular design enables interpretable, robust, and reproducible Simulink simulation generation. Our source code is publicly available at https://github.com/renxinxing123/SimuGen_beta.

Problem

• Generating reliable Simulink models from text inputs
• Addressing LLMs' lack of Simulink-specific pretraining data
• Automating accurate simulation code through multimodal framework

Innovation

• Multimodal agent framework for Simulink code generation
• Visual diagram and domain knowledge integration approach
• Collaborative specialized agents enabling robust simulation generation