Efficient Exploration Is Enough

2026-09-07

AI Summary

本文研究在无外部奖励情况下，通过优先生成可泛化的经验来实现高效探索，理论上和实证上展示了该方法能自动产生复杂行为。

Abstract

This work introduces an alternative view of efficient exploration and studies its theoretical and empirical implications in the absence of extrinsic rewards. Specifically, we define efficient explorers as agents that prioritize generating generalizable experience, i.e., data that supports learning models capable of predicting and adapting across the environment. This allows us to analyze efficient exploration through the lens of prediction and generalization. Theoretically, we demonstrate that optimally efficient explorers naturally schedule their trajectories to visit the most informative and learnable regions first. Empirically, we show that optimizing for these agents gives rise to an automatic curriculum of progressively more complex behaviors, even in relatively simple environments. These results indicate that pursuing this purely intrinsic objective alone is enough to drive the emergence of highly sophisticated behaviors. We believe that this new framework provides a principled mechanism by which agent-environment systems may sustain an open-ended process of increasingly complex behavior without external rewards, tasks, or objectives.

Problem

• efficient exploration
• intrinsic objective
• generalization
• prediction
• open-ended process

Innovation

• efficient exploration
• generalizable experience
• intrinsic objective