Scaling Multi-Reference Image Generation with Dynamic Reward Optimization

2026-06-25

AI Summary

This work addresses the significant performance degradation of multi-reference image generation (MRIG) in complex scenarios, where existing open-source models struggle to effectively integrate numerous heterogeneous reference images. To overcome this limitation, the authors propose DyRef, a two-stage training framework that first establishes foundational MRIG capabilities through supervised fine-tuning and then enhances policy learning via dynamic optimization of the reinforcement learning objective. This is achieved by introducing Difficulty-Aware Advantage Reweighting (DAR) and Discriminative Reward Scaling (DRS) mechanisms. Additionally, the study introduces OmniRef-Bench, a comprehensive benchmark encompassing diverse and complex reference combinations. Experimental results demonstrate that DyRef substantially improves the performance of open-source models on both OmniRef-Bench and single-image editing tasks, confirming its effectiveness and strong generalization capability.

Abstract

While personalized image generation has achieved remarkable progress, multi-reference image generation (MRIG) remains a challenging task. Most existing benchmarks fail to adequately evaluate complex MRIG scenarios, hindering further progress in this area. To better assess model performance on complex MRIG tasks, we introduce OmniRef-Bench, a benchmark that covers complex combinations of reference image types and a large number of reference images. Evaluations on OmniRef-Bench show that mainstream open-source models struggle in complex MRIG scenarios, and their performance deteriorates significantly as the number of mixed-type reference images increases. To address this issue, we propose DyRef, a two-stage training framework. In the first stage, supervised fine-tuning equips the model with the basic capability to handle complex MRIG tasks. In the second stage, we introduce Difficulty-aware Advantage Reweighting (DAR) and Discriminative Reward Scaling (DRS). DAR dynamically adjusts the optimization objective to improve performance when handling a large number of mixed-type reference images. DRS enlarges intra-group reward differences for more effective policy optimization. Experiments demonstrate that DyRef significantly improves the performance of open-source models on OmniRef-Bench and single-image editing benchmarks, demonstrating the effectiveness and generalization capability of our approach.

Problem

• multi-reference image generation
• complex MRIG scenarios
• benchmark evaluation
• mixed-type reference images

Innovation

• multi-reference image generation
• dynamic reward optimization
• OmniRef-Bench
• Difficulty-aware Advantage Reweighting
• Discriminative Reward Scaling