From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems

2025-06-05

AI Summary

Current research on Composite Artificial Intelligence Systems (CAIS) is fragmented, lacking a unified analytical and evaluation framework. To address this, we systematically survey synergistic paradigms among large language models (LLMs), retrievers, tools, agents, and orchestrators; propose the first multidimensional CAIS taxonomy—explicitly defining four foundational architectures: Retrieval-Augmented Generation (RAG), LLM-based agents, multimodal foundation models, and orchestration-centric systems; and develop a cross-paradigm design trade-off model alongside a system-level evaluation framework that integrates modular design, dynamic scheduling, and heterogeneous component coordination. Our work establishes the first comprehensive survey framework covering the full CAIS spectrum, uncovering core challenges—including scalability, interoperability, and benchmark development—and providing theoretical foundations and practical guidelines for next-generation system-level AI.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent/AI Theories and Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Compound Al Systems (CAIS) is an emerging paradigm that integrates large language models (LLMs) with external components, such as retrievers, agents, tools, and orchestrators, to overcome the limitations of standalone models in tasks requiring memory, reasoning, real-time grounding, and multimodal understanding. These systems enable more capable and context-aware behaviors by composing multiple specialized modules into cohesive workflows. Despite growing adoption in both academia and industry, the CAIS landscape remains fragmented, lacking a unified framework for analysis, taxonomy, and evaluation. In this survey, we define the concept of CAIS, propose a multi-dimensional taxonomy based on component roles and orchestration strategies, and analyze four foundational paradigms: Retrieval-Augmented Generation (RAG), LLM Agents, Multimodal LLMs (MLLMs), and orchestration-centric architectures. We review representative systems, compare design trade-offs, and summarize evaluation methodologies across these paradigms. Finally, we identify key challenges-including scalability, interoperability, benchmarking, and coordination-and outline promising directions for future research. This survey aims to provide researchers and practitioners with a comprehensive foundation for understanding, developing, and advancing the next generation of system-level artificial intelligence.

Problem

• Overcoming standalone LLM limitations via integrated components
• Addressing fragmentation in CAIS analysis and taxonomy
• Enhancing AI system scalability and interoperability challenges

Innovation

• Integrates LLMs with retrievers, agents, tools
• Proposes multi-dimensional taxonomy for CAIS
• Analyzes RAG, LLM Agents, MLLMs, orchestration