Goal Recognition using Actor-Critic Optimization

2024-12-31

AI Summary

Existing approaches to agent goal inference from unstructured observation sequences rely on hand-crafted rules and discrete symbolic representations, limiting their accuracy and generalizability in complex, continuous environments. Method: We propose DRACO, the first end-to-end deep reinforcement learning framework for this task. It eliminates domain knowledge and explicit symbolic representations, learning multi-strategy goal inference directly from raw observations. Contributions/Results: (1) We introduce a novel hypothesis evaluation metric based on continuous policy representations, enabling tight integration of goal recognition and deep RL; (2) leveraging an Actor-Critic architecture with policy network distillation, DRACO achieves state-of-the-art performance in discrete domains and significantly outperforms baselines in continuous settings; (3) it reduces computational and memory overhead substantially while demonstrating strong robustness and cross-domain generalization capability.

Technology Category

• Multiagent Systems: Adversarial Agents
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Goal Recognition aims to infer an agent's goal from a sequence of observations. Existing approaches often rely on manually engineered domains and discrete representations. Deep Recognition using Actor-Critic Optimization (DRACO) is a novel approach based on deep reinforcement learning that overcomes these limitations by providing two key contributions. First, it is the first goal recognition algorithm that learns a set of policy networks from unstructured data and uses them for inference. Second, DRACO introduces new metrics for assessing goal hypotheses through continuous policy representations. DRACO achieves state-of-the-art performance for goal recognition in discrete settings while not using the structured inputs used by existing approaches. Moreover, it outperforms these approaches in more challenging, continuous settings at substantially reduced costs in both computing and memory. Together, these results showcase the robustness of the new algorithm, bridging traditional goal recognition and deep reinforcement learning.

Problem

• Object Recognition
• Complex Data
• Artificial Intelligence

Innovation

• DRACO
• Deep Learning Target Recognition
• Multi-strategy Learning