A Survey of Large Language Model Agents for Question Answering

2025-03-24

AI Summary

Traditional question-answering (QA) systems suffer from strong data dependency and poor adaptability to dynamic environments. Method: This paper proposes a large language model (LLM)-based agent architecture for QA, introducing the first hierarchical LLM agent framework specifically designed for QA tasks. It comprises four stages—planning, question understanding, information retrieval, and answer generation—and integrates multi-step task decomposition, external tool invocation, and retrieval-augmented generation (RAG). Contribution/Results: The framework significantly enhances interactive reasoning capabilities and cross-environment generalization compared to conventional pipeline-based and naive LLM-based QA approaches. Through systematic evaluation, we identify key performance bottlenecks and distill three critical future research directions: scalability, trustworthy reasoning, and environment-aware collaboration. This work provides both theoretical foundations and a structured roadmap for advancing LLM-agent-driven QA systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Natural Language Processing: Question Answering
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

This paper surveys the development of large language model (LLM)-based agents for question answering (QA). Traditional agents face significant limitations, including substantial data requirements and difficulty in generalizing to new environments. LLM-based agents address these challenges by leveraging LLMs as their core reasoning engine. These agents achieve superior QA results compared to traditional QA pipelines and naive LLM QA systems by enabling interaction with external environments. We systematically review the design of LLM agents in the context of QA tasks, organizing our discussion across key stages: planning, question understanding, information retrieval, and answer generation. Additionally, this paper identifies ongoing challenges and explores future research directions to enhance the performance of LLM agent QA systems.

Problem

• Surveying LLM-based agents for QA improvements
• Addressing data and generalization limits in QA agents
• Exploring LLM agents' design stages for QA tasks

Innovation

• LLM-based agents as core reasoning engine
• Interaction with external environments for QA
• Systematic review of LLM agent design stages