Zero-Shot Action Generalization with Limited Observations

2025-03-11

AI Summary

Reinforcement learning agents struggle with zero-shot generalization to unseen actions when provided with only 1–3 observations of novel actions. Method: This paper proposes AGLO, a framework that jointly optimizes action semantic modeling and policy transfer via decoupled action representation learning—leveraging contrastive embedding and synthetic augmentation—and policy learning via knowledge distillation. Contribution/Results: AGLO is the first method to achieve robust zero-shot action generalization under extremely limited action observations, eliminating reliance on large-scale action datasets. Evaluated across multiple benchmark tasks, AGLO significantly outperforms state-of-the-art approaches, improving action generalization accuracy by an average of 27.4%, thereby demonstrating both effectiveness and practical applicability.

Technology Category

• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Reinforcement Learning (RL) has demonstrated remarkable success in solving sequential decision-making problems. However, in real-world scenarios, RL agents often struggle to generalize when faced with unseen actions that were not encountered during training. Some previous works on zero-shot action generalization rely on large datasets of action observations to capture the behaviors of new actions, making them impractical for real-world applications. In this paper, we introduce a novel zero-shot framework, Action Generalization from Limited Observations (AGLO). Our framework has two main components: an action representation learning module and a policy learning module. The action representation learning module extracts discriminative embeddings of actions from limited observations, while the policy learning module leverages the learned action representations, along with augmented synthetic action representations, to learn a policy capable of handling tasks with unseen actions. The experimental results demonstrate that our framework significantly outperforms state-of-the-art methods for zero-shot action generalization across multiple benchmark tasks, showcasing its effectiveness in generalizing to new actions with minimal action observations.

Problem

• Generalizing to unseen actions in RL with limited observations.
• Learning action representations from minimal data for zero-shot tasks.
• Improving policy learning for handling new actions without extensive datasets.

Innovation

• AGLO framework for zero-shot action generalization
• Action representation learning from limited observations
• Policy learning with synthetic action representations