Behavioral Exploration: Learning to Explore via In-Context Adaptation

2025-07-11

AI Summary

This work addresses the limited capability of autonomous agents to rapidly explore unknown environments and adapt behaviors online. We propose a behavior exploration framework that integrates contextual learning with behavioral cloning. Leveraging long-context generative models, the framework internalizes exploration strategies, enabling agents to dynamically select and switch among expert behavioral policies based on historical interaction context—thereby achieving goal-directed, efficient, human-like exploration. Its core innovation lies in synergistically combining the generalization capacity of behavioral cloning with the rapid adaptability of in-context learning, allowing complex tasks to be accomplished with minimal interaction. Extensive evaluation on both simulated and real-world robotic manipulation tasks demonstrates substantial improvements in environmental adaptability and exploration efficiency over state-of-the-art online adaptation methods.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Developing autonomous agents that quickly explore an environment and adapt their behavior online is a canonical challenge in robotics and machine learning. While humans are able to achieve such fast online exploration and adaptation, often acquiring new information and skills in only a handful of interactions, existing algorithmic approaches tend to rely on random exploration and slow, gradient-based behavior updates. How can we endow autonomous agents with such capabilities on par with humans? Taking inspiration from recent progress on both in-context learning and large-scale behavioral cloning, in this work we propose behavioral exploration: training agents to internalize what it means to explore and adapt in-context over the space of ``expert'' behaviors. To achieve this, given access to a dataset of expert demonstrations, we train a long-context generative model to predict expert actions conditioned on a context of past observations and a measure of how ``exploratory'' the expert's behaviors are relative to this context. This enables the model to not only mimic the behavior of an expert, but also, by feeding its past history of interactions into its context, to select different expert behaviors than what have been previously selected, thereby allowing for fast online adaptation and targeted, ``expert-like'' exploration. We demonstrate the effectiveness of our method in both simulated locomotion and manipulation settings, as well as on real-world robotic manipulation tasks, illustrating its ability to learn adaptive, exploratory behavior.

Problem

• Develop autonomous agents for fast online exploration and adaptation
• Enable agents to mimic and adapt expert behaviors contextually
• Improve robotic exploration and adaptation in simulated and real-world tasks

Innovation

• Training agents to explore via in-context adaptation
• Using long-context generative models for expert behavior prediction
• Enabling fast online adaptation and expert-like exploration