A Survey on Large Language Model based Human-Agent Systems

2025-05-01

AI Summary

To address inherent limitations of LLM-driven fully autonomous agents—particularly regarding reliability, safety, complex task execution, and ethical compliance—this paper proposes a human–AI collaboration enhancement paradigm. We systematically design the first unified framework for Large Language Model–Human Augmented Systems (LLM-HAS), formally defining core components including environment modeling, interaction modalities, and orchestration mechanisms. By integrating human feedback, real-time human intervention, and domain-specific knowledge, our approach significantly improves agent task performance and trustworthiness. Methodologically, the work synthesizes principles from human–computer interaction design, feedback-driven learning, multi-agent orchestration, and trustworthy AI evaluation. As key contributions, we establish the first structured survey taxonomy for LLM-HAS, release an open-source, high-quality paper repository and resource catalog, and provide foundational theory, methodological guidelines, and practical tooling to advance research and deployment of human–AI collaborative systems.

Technology Category

• Humans and AI: Human-AI Collaboration / Human-AI Teaming
• Multiagent Systems: Coordination and Collaboration
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Recent advances in large language models (LLMs) have sparked growing interest in building fully autonomous agents. However, fully autonomous LLM-based agents still face significant challenges, including limited reliability due to hallucinations, difficulty in handling complex tasks, and substantial safety and ethical risks, all of which limit their feasibility and trustworthiness in real-world applications. To overcome these limitations, LLM-based human-agent systems (LLM-HAS) incorporate human-provided information, feedback, or control into the agent system to enhance system performance, reliability and safety. This paper provides the first comprehensive and structured survey of LLM-HAS. It clarifies fundamental concepts, systematically presents core components shaping these systems, including environment&profiling, human feedback, interaction types, orchestration and communication, explores emerging applications, and discusses unique challenges and opportunities. By consolidating current knowledge and offering a structured overview, we aim to foster further research and innovation in this rapidly evolving interdisciplinary field. Paper lists and resources are available at https://github.com/HenryPengZou/Awesome-LLM-Based-Human-Agent-System-Papers.

Problem

• Overcoming reliability and safety challenges in autonomous LLM agents
• Enhancing LLM agent performance with human feedback and control
• Surveying LLM-human agent systems for interdisciplinary research advancement

Innovation

• Incorporates human feedback for reliability
• Systematic survey of LLM-human-agent systems
• Explores interaction types and orchestration