MARCO: A Multi-Agent System for Optimizing HPC Code Generation Using Large Language Models

2025-05-06

AI Summary

Large language models (LLMs) often neglect critical high-performance computing (HPC) optimization requirements—including parallelism, memory efficiency, and hardware-aware architecture adaptation—when generating HPC code. Method: This paper proposes MARCO, a Multi-Agent Reactive Code Optimizer, featuring (i) a closed-loop feedback mechanism between a code generation agent and a performance evaluation agent for dynamic code refinement; (ii) a novel real-time web retrieval module that dynamically incorporates state-of-the-art optimization strategies from recent HPC conference publications to mitigate LLM knowledge staleness; and (iii) a lightweight, scalable multi-agent collaboration paradigm. Results: On the LeetCode 75 benchmark, MARCO achieves an average 14.6% speedup over Claude 3.5 Sonnet; integrating web retrieval further improves runtime efficiency by 30.9% relative to the baseline.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models (LLMs) have transformed software development through code generation capabilities, yet their effectiveness for high-performance computing (HPC) remains limited. HPC code requires specialized optimizations for parallelism, memory efficiency, and architecture-specific considerations that general-purpose LLMs often overlook. We present MARCO (Multi-Agent Reactive Code Optimizer), a novel framework that enhances LLM-generated code for HPC through a specialized multi-agent architecture. MARCO employs separate agents for code generation and performance evaluation, connected by a feedback loop that progressively refines optimizations. A key innovation is MARCO's web-search component that retrieves real-time optimization techniques from recent conference proceedings and research publications, bridging the knowledge gap in pre-trained LLMs. Our extensive evaluation on the LeetCode 75 problem set demonstrates that MARCO achieves a 14.6% average runtime reduction compared to Claude 3.5 Sonnet alone, while the integration of the web-search component yields a 30.9% performance improvement over the base MARCO system. These results highlight the potential of multi-agent systems to address the specialized requirements of high-performance code generation, offering a cost-effective alternative to domain-specific model fine-tuning.

Problem

• Optimizing HPC code generation using multi-agent LLM systems
• Addressing specialized HPC needs like parallelism and memory efficiency
• Bridging knowledge gaps in LLMs with real-time web-search techniques

Innovation

• Multi-agent system for HPC code optimization
• Feedback loop refines code generation progressively
• Web-search retrieves real-time optimization techniques