Analyzing Code Injection Attacks on LLM-based Multi-Agent Systems in Software Development

2025-12-25

AI Summary

This study identifies a critical security vulnerability in LLM-driven multi-agent software development systems (e.g., coder-reviewer-tester architectures): their high autonomy and lack of intrinsic safety mechanisms render them susceptible to code injection attacks. We first establish a fine-grained threat model tailored to multi-agent software pipelines. To address this, we propose a novel defense paradigm integrating a dedicated security analysis agent—enabling robust protection without compromising development efficiency. Empirical evaluation reveals that few-shot poisoning injections increase attack success rates from 0% to 71.95%. Experiments further demonstrate that the coder-reviewer-tester architecture exhibits superior robustness over coder-only or coder-tester variants; the security agent effectively reconciles efficient code generation with strong adversarial resilience; and advanced adversarial injection attacks are successfully reproduced and quantified. Our core contributions include: (1) a novel, pipeline-aware threat modeling framework; (2) a principled security agent architecture; and (3) empirical validation and characterization of poisoning-based injection attacks.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Agentic AI and Multi-Agent Systems are poised to dominate industry and society imminently. Powered by goal-driven autonomy, they represent a powerful form of generative AI, marking a transition from reactive content generation into proactive multitasking capabilities. As an exemplar, we propose an architecture of a multi-agent system for the implementation phase of the software engineering process. We also present a comprehensive threat model for the proposed system. We demonstrate that while such systems can generate code quite accurately, they are vulnerable to attacks, including code injection. Due to their autonomous design and lack of humans in the loop, these systems cannot identify and respond to attacks by themselves. This paper analyzes the vulnerability of multi-agent systems and concludes that the coder-reviewer-tester architecture is more resilient than both the coder and coder-tester architectures, but is less efficient at writing code. We find that by adding a security analysis agent, we mitigate the loss in efficiency while achieving even better resiliency. We conclude by demonstrating that the security analysis agent is vulnerable to advanced code injection attacks, showing that embedding poisonous few-shot examples in the injected code can increase the attack success rate from 0% to 71.95%.

Problem

• Analyzes code injection vulnerabilities in LLM-based multi-agent systems
• Compares resilience and efficiency of different software development agent architectures
• Demonstrates advanced attacks exploiting few-shot examples in injected code

Innovation

• Proposed coder-reviewer-tester multi-agent architecture for software implementation
• Added security analysis agent to improve resilience and efficiency
• Analyzed vulnerability to code injection via poisoned few-shot examples