Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents

2024-11-05

AI Summary

LLM-based agents suffer from unobservable and hard-to-debug reasoning errors due to implicit, fast “System 1”-style inference. Method: This paper proposes Watson, a cognitive observability framework that enables non-intrusive, dynamic inversion of agent reasoning traces and semantic-level error localization—without modifying model architecture or training pipelines. Watson integrates reasoning trajectory reconstruction, cognitive state modeling, and context-aware error correction, synergizing program analysis with prompt engineering. Contribution/Results: Watson supports real-time error detection, root-cause attribution, and automated correction. Experiments show absolute Pass@1 improvements of +7.58 and +7.76 percentage points (relative gains of 13.45% and 12.31%) on MMLU and SWE-bench-lite, respectively, significantly enhancing the debuggability and robustness of LLM agents.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

As foundation models (FMs) play an increasingly prominent role in complex software systems, such as agentic software, they introduce significant observability and debuggability challenges. Although recent Large Reasoning Models (LRMs) generate their thought processes as part of the output, in many scenarios fast-thinking Large Language Models (LLMs) are still preferred due to latency constraints. LLM-powered agents operate autonomously with opaque implicit reasoning, making it difficult to debug their unexpected behaviors or errors. In this paper, we introduce Watson, a novel framework that provides reasoning observability into the implicit reasoning processes of agents driven by fast-thinking LLMs, allowing the identification and localization of errors and guidance for corrections. We demonstrate the accuracy of the recovered implicit reasoning trace by Watson and its usefulness through debugging and improving the performance of LLM-powered agents in two scenarios: Massive Multitask Language Understanding (MMLU) benchmark and SWE-bench-lite. Using Watson, we were able to observe and identify the implicit reasoning errors, and automatically provide targeted corrections at runtime that improve the Pass@1 of agents on MMLU and SWE-bench-lite by 7.58 (13.45% relative improvement) and 7.76 (12.31% relative improvement) percentage points, respectively, without updates to models or the cognitive architecture of the agents.

Problem

• Addresses observability challenges in LLM-powered agents.
• Provides insights into implicit reasoning processes of fast-thinking LLMs.
• Enables debugging and performance improvement in autonomous agents.

Innovation

• Watson framework enhances LLM reasoning observability.
• Identifies and localizes errors in LLM-powered agents.
• Automatically provides targeted corrections at runtime.