Bridging VLMs and Embodied Intelligence with Deliberate Practice Policy Optimization

2025-11-20

AI Summary

Embodied intelligence faces dual challenges of scarce real-world data and low algorithmic efficiency. To address these, we propose Deliberate Practice Policy Optimization (DPPO), the first metacognition-driven “Metaloop” training framework. DPPO dynamically alternates between supervised fine-tuning and reinforcement learning, while automatically identifying capability gaps and reallocating computational resources toward underperforming skills—enabling efficient learning from sparse embodied data. Furthermore, we integrate vision-language modeling with embodied decision-making to introduce Pelican-VL 1.0, a unified multimodal foundation model. Experiments demonstrate that Pelican-VL 1.0 achieves a 20.3% absolute improvement over strong baselines on standard embodied tasks and outperforms open-source billion-parameter models by 10.6%. All code and models are publicly released, establishing a new paradigm for efficient, scalable development of embodied agents.

Technology Category

• Intelligent Robots: Embodied AI
• Computer Vision: Large Vision Models
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Developing a universal and versatile embodied intelligence system presents two primary challenges: the critical embodied data bottleneck, where real-world data is scarce and expensive, and the algorithmic inefficiency of existing methods, which are resource-prohibitive. To address these limitations, we introduce Deliberate Practice Policy Optimization (DPPO), a metacognitive ``Metaloop'' training framework that dynamically alternates between supervised fine-tuning (competence expansion) and reinforcement learning (skill refinement). This enables automatic weakness identification and targeted resource allocation, specifically designed to maximize learning efficiency from sparse, finite data. Theoretically, DPPO can be formalised as a unified preference-learning framework. Empirically, training a vision-language embodied model with DPPO, referred to as Pelican-VL 1.0, yields a 20.3% performance improvement over the base model and surpasses open-source models at the 100B-parameter scale by 10.6%. We are open-sourcing both the models and code, providing the first systematic framework that alleviates the data and resource bottleneck and enables the community to build versatile embodied agents efficiently.

Problem

• Overcoming scarce real-world data for embodied intelligence systems
• Improving algorithmic inefficiency in resource-intensive training methods
• Developing unified framework for weakness identification and targeted learning

Innovation

• Metacognitive Metaloop alternates SFT and RL
• Automatic weakness identification maximizes sparse data learning
• Unified preference-learning framework formalizes DPPO theoretically