MuaLLM: A Multimodal Large Language Model Agent for Circuit Design Assistance with Hybrid Contextual Retrieval-Augmented Generation

2025-08-11

AI Summary

Circuit design research faces mounting challenges—including rapidly expanding literature, heterogeneous data representations, and complex multi-objective optimization—leading to inefficient literature review. To address these, we propose a multimodal large language model (MLLM)-based agent framework tailored for circuit design. Our method innovatively integrates a Reason+Act iterative reasoning workflow with hybrid retrieval-augmented generation (RAG), decoupling retrieval from reasoning to overcome context-length limitations and enable scalable literature expansion and automated knowledge base construction. The framework unifies an MLLM, an adaptive vector database, intelligent search tools, and real-time web crawling for joint image-text analysis. Evaluated on the RAG-250 and Reas-100 benchmarks, it achieves 90.1% recall and 86.8% multi-step reasoning accuracy, respectively, while accelerating inference by 1.6× and reducing computational cost to one-tenth that of standard LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Conducting a comprehensive literature review is crucial for advancing circuit design methodologies. However, the rapid influx of state-of-the-art research, inconsistent data representation, and the complexity of optimizing circuit design objectives make this task significantly challenging. In this paper, we propose MuaLLM, an open-source multimodal Large Language Model (LLM) agent for circuit design assistance that integrates a hybrid Retrieval-Augmented Generation (RAG) framework with an adaptive vector database of circuit design research papers. Unlike conventional LLMs, the MuaLLM agent employs a Reason + Act (ReAct) workflow for iterative reasoning, goal-setting, and multi-step information retrieval. It functions as a question-answering design assistant, capable of interpreting complex queries and providing reasoned responses grounded in circuit literature. Its multimodal capabilities enable processing of both textual and visual data, facilitating more efficient and comprehensive analysis. The system dynamically adapts using intelligent search tools, automated document retrieval from the internet, and real-time database updates. Unlike conventional approaches constrained by model context limits, MuaLLM decouples retrieval from inference, enabling scalable reasoning over arbitrarily large corpora. At the maximum context length supported by standard LLMs, MuaLLM remains up to 10x less costly and 1.6x faster while maintaining the same accuracy. This allows rapid, no-human-in-the-loop database generation, overcoming the bottleneck of simulation-based dataset creation for circuits. To evaluate MuaLLM, we introduce two custom datasets: RAG-250, targeting retrieval and citation performance, and Reasoning-100 (Reas-100), focused on multistep reasoning in circuit design. MuaLLM achieves 90.1% recall on RAG-250, and 86.8% accuracy on Reas-100.

Problem

• Addressing challenges in circuit design literature review
• Enhancing multimodal data processing for circuit analysis
• Overcoming context limits in scalable reasoning systems

Innovation

• Hybrid Retrieval-Augmented Generation framework
• Multimodal processing of text and visuals
• Dynamic adaptation with intelligent search tools