Curriculum Group Policy Optimization: Adaptive Sampling for Unleashing the Potential of Text-to-Image Generation

2026-05-17

AI Summary

This work addresses the inefficiency of uniform sampling in reinforcement learning–based text-to-image generation, which disregards the alignment between sample difficulty and model capability. To overcome this limitation, the authors propose Curriculum Group Policy Optimization (CGPO), the first framework to integrate curriculum learning into group policy optimization. CGPO dynamically adjusts prompt sampling probabilities by using the online-estimated reward variance of generated images as a proxy for difficulty, prioritizing prompts that the model partially grasps but has not yet mastered. Additionally, it introduces a category calibration mechanism based on proportional-fairness optimization to mitigate data imbalance across multiple categories. Experiments on GenEval, T2I-CompBench++, and DPG Bench demonstrate that CGPO substantially improves both generation quality and training efficiency.

Abstract

Text-to-Image (T2I) generation has achieved remarkable progress in recent years. Meanwhile, reinforcement learning methods, particularly those based on Group Relative Policy Optimization (GRPO), have attracted widespread attention and been successfully applied to T2I tasks. However, the uniform sampling strategy commonly used during training often ignores the match between sample difficulty and the model's current learning capability, leading to low training efficiency. We argue that improving training efficiency requires continuously prioritizing prompts that match the model's evolving capability and remain actively learnable. To this end, we propose Curriculum Group Policy Optimization (CGPO), an adaptive curriculum training framework. During training, each prompt produces a group of images scored by a reward model. We use the variance of group rewards as an online proxy for prompt inconsistency. A higher variance suggests that the model has partially captured the prompt requirements but has not yet achieved stable mastery. Such prompts are more likely to provide useful learning signals, so we increase their sampling probabilities accordingly. Additionally, to address data imbalance in multi-category datasets, we design a category calibration method based on proportional fairness optimization, which balances training difficulty across categories. Experiments on GenEval, T2I-CompBench++, and DPG Bench demonstrate that our framework effectively improves generation performance.

Problem

• Text-to-Image Generation
• Training Efficiency
• Sample Difficulty
• Reinforcement Learning
• Curriculum Learning

Innovation

• Curriculum Learning
• Group Relative Policy Optimization
• Adaptive Sampling
• Reward Variance
• Proportional Fairness