Generated, Parallel, Scalable? A Study of Agentic AI-Generated Julia Code on Supercomputers

2026-06-15

AI Summary

This work investigates the capability of agent-based AI systems to automatically generate scalable, high-performance parallel Julia code, aiming to lower the barrier to parallel programming in high-performance computing. We develop a system integrating the OpenCode agent framework with a Julia documentation MCP server and present the first systematic evaluation of large language models within an agent paradigm for generating and optimizing code across task-based parallelism (Dagger.jl) and conventional backends (Base.Threads, MPI.jl). Experiments span shared- and distributed-memory architectures, evaluating models including GPT-5.5, Claude Opus 4.7, and Qwen3-Coder-Next. Results show that agents can produce executable code for small-scale problems but encounter significant challenges—such as deadlocks, over-subscription, and out-of-memory errors—at larger scales. Commercial models generally outperform open-source counterparts, yet all exhibit notable deficiencies in modeling task dependencies and scheduling.

Abstract

Julia is increasingly used in hpc as a single-language alternative to combining high-level scripting with low-level systems languages, but achieving scalable performance still requires expertise in parallel programming. llms are increasingly used for code generation and are advancing rapidly with each new version. Yet, existing studies focus on single-shot prompting rather than agentic settings, in which an llm autonomously plans, generates, and refines code through tool use. Using an OpenCode-based agent extended with a Julia-documentation mcp server, we study agentic generation of parallel Julia code, focusing on task-based execution with Dagger.jl. We evaluate three llms, OpenAI GPT-5.5, Anthropic Claude Opus 4.7, and the open-weight Qwen3-Coder-Next, on three problems with distinct parallel structures: π approximation, tiled general matrix multiplication, and tiled Cholesky decomposition. The generated Dagger.jl implementations are compared against agent-generated Base.Threads and MPI.jl baselines, with shared-memory experiments scaling to 192 cores and distributed-memory experiments on two nodes. The agents reliably produce executable code for small inputs but fail at larger scales due to deadlocks, oversubscription, or out-of-memory errors, with the open-weight model affected most severely. The two commercial models scale comparably on Base.Threads and MPI.jl, while their Dagger.jl implementations expose recurring weaknesses in task dependencies, granularity, and scheduling. Agentic AI is promising for producing parallel Julia code, but generating robust, performance-aware implementations for large-scale hpc systems remains an open challenge.

Problem

• Agentic AI
• Parallel Code Generation
• Julia
• Scalability
• High-Performance Computing

Innovation

• Agentic AI
• Parallel Code Generation
• Julia
• Dagger.jl
• Scalable HPC