ReflAct: World-Grounded Decision Making in LLM Agents via Goal-State Reflection

2025-05-21

AI Summary

Existing LLM agents (e.g., ReAct) suffer from goal misalignment and hallucination in complex environments due to reasoning drift from the actual world state—a consequence of insufficient maintenance of internal belief–goal consistency. To address this, we propose the “Goal–State Reflection” decision framework, which shifts the reasoning focus from action planning to dynamic, iterative reflection on the alignment between the current world state and the task goal. Our method explicitly models the world state, quantifies goal deviation, and generates self-correcting reasoning chains to ensure continuous state awareness and goal fidelity. Evaluated on ALFWorld, our approach achieves a 93.3% task success rate—outperforming standard ReAct by +27.7% and surpassing enhanced variants incorporating Reflexion or World Knowledge Memory (WKM). This work establishes a novel paradigm for improving the strategic reliability of LLM-based agents through persistent goal–state coherence.

Technology Category

• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Knowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advances in LLM agents have largely built on reasoning backbones like ReAct, which interleave thought and action in complex environments. However, ReAct often produces ungrounded or incoherent reasoning steps, leading to misalignment between the agent's actual state and goal. Our analysis finds that this stems from ReAct's inability to maintain consistent internal beliefs and goal alignment, causing compounding errors and hallucinations. To address this, we introduce ReflAct, a novel backbone that shifts reasoning from merely planning next actions to continuously reflecting on the agent's state relative to its goal. By explicitly grounding decisions in states and enforcing ongoing goal alignment, ReflAct dramatically improves strategic reliability. This design delivers substantial empirical gains: ReflAct surpasses ReAct by 27.7% on average, achieving a 93.3% success rate in ALFWorld. Notably, ReflAct even outperforms ReAct with added enhancement modules (e.g., Reflexion, WKM), showing that strengthening the core reasoning backbone is key to reliable agent performance.

Problem

• Improving grounded reasoning in LLM agents
• Reducing misalignment between agent state and goal
• Enhancing strategic reliability via goal-state reflection

Innovation

• ReflAct enhances decision-making via goal-state reflection
• Grounds decisions in states for strategic reliability
• Outperforms ReAct by 27.7% in ALFWorld