VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning

2025-07-30

AI Summary

Multimodal reasoning models exhibit unstable performance across diverse domains and difficulty levels. Method: This paper proposes Progressive Curriculum Reinforcement Learning (PCuRL), a framework that enables adaptive difficulty scheduling and controllable reasoning path length optimization via online soft difficulty weighting and a dynamic-length reward mechanism. PCuRL integrates vision-language modeling with reinforcement learning to support hierarchical, curriculum-based multimodal reasoning—from simple to complex tasks. Contribution/Results: The method significantly improves reasoning stability and generalization across mathematical, scientific, logical, and general comprehension tasks. Extensive experiments demonstrate consistent performance on mainstream multimodal benchmarks—matching or surpassing state-of-the-art models—thereby validating PCuRL’s effectiveness and robustness.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Reinforcement learning has proven its effectiveness in enhancing the reasoning capabilities of large language models. Recent research efforts have progressively extended this paradigm to multimodal reasoning tasks. Due to the inherent complexity and diversity of multimodal tasks, especially in semantic content and problem formulations, existing models often exhibit unstable performance across various domains and difficulty levels. To address these limitations, we propose VL-Cogito, an advanced multimodal reasoning model trained via a novel multi-stage Progressive Curriculum Reinforcement Learning (PCuRL) framework. PCuRL systematically guides the model through tasks of gradually increasing difficulty, substantially improving its reasoning abilities across diverse multimodal contexts. The framework introduces two key innovations: (1) an online difficulty soft weighting mechanism, dynamically adjusting training difficulty across successive RL training stages; and (2) a dynamic length reward mechanism, which encourages the model to adaptively regulate its reasoning path length according to task complexity, thus balancing reasoning efficiency with correctness. Experimental evaluations demonstrate that VL-Cogito consistently matches or surpasses existing reasoning-oriented models across mainstream multimodal benchmarks spanning mathematics, science, logic, and general understanding, validating the effectiveness of our approach.

Problem

• Enhance multimodal reasoning across diverse domains
• Stabilize performance in varying difficulty levels
• Balance reasoning efficiency with correctness adaptively

Innovation

• Progressive Curriculum Reinforcement Learning framework
• Online difficulty soft weighting mechanism
• Dynamic length reward mechanism