IMLE Policy: Fast and Sample Efficient Visuomotor Policy Learning via Implicit Maximum Likelihood Estimation

2025-02-17

AI Summary

To address the high data cost and poor real-time performance in robotic imitation learning, this paper proposes a one-step vision-motion policy learning method based on Implicit Maximum Likelihood Estimation (IMLE). We are the first to integrate IMLE into the behavior cloning framework, designing a lightweight generative action generator that enables single-step inference and multimodal behavior modeling while significantly reducing data dependency—achieving an average 38% reduction in required demonstration samples. Compared to Diffusion Policy, our method improves inference speed by 97.3%; it also outperforms single-step Flow Matching in policy performance. Extensive experiments on both simulated and real-world robotic arm tasks validate its efficiency, generalization capability, and strong low-data adaptation performance.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Behavior Learning & Control
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Recent advances in imitation learning, particularly using generative modelling techniques like diffusion, have enabled policies to capture complex multi-modal action distributions. However, these methods often require large datasets and multiple inference steps for action generation, posing challenges in robotics where the cost for data collection is high and computation resources are limited. To address this, we introduce IMLE Policy, a novel behaviour cloning approach based on Implicit Maximum Likelihood Estimation (IMLE). IMLE Policy excels in low-data regimes, effectively learning from minimal demonstrations and requiring 38% less data on average to match the performance of baseline methods in learning complex multi-modal behaviours. Its simple generator-based architecture enables single-step action generation, improving inference speed by 97.3% compared to Diffusion Policy, while outperforming single-step Flow Matching. We validate our approach across diverse manipulation tasks in simulated and real-world environments, showcasing its ability to capture complex behaviours under data constraints. Videos and code are provided on our project page: https://imle-policy.github.io/.

Problem

• Efficient policy learning with minimal data
• Single-step action generation for speed
• Handling complex multi-modal behaviours effectively

Innovation

• Implicit Maximum Likelihood Estimation
• Single-step action generation
• Low-data regime efficiency