Divide-and-Denoise: A Game-Theoretic Method for Fairly Composing Diffusion Models

2026-06-08

AI Summary

This work addresses the challenge that combining multiple pre-trained diffusion models often leads to missing objects or attribute mismatches due to dominance or conflict among the models. To mitigate this, the authors propose a dynamic task allocation mechanism grounded in fair division game theory, which dynamically partitions responsibility regions at each denoising step to guide collaborative denoising within assigned areas. By deeply integrating fair division algorithms from game theory with the diffusion sampling process, the method enables efficient and equitable co-evolutionary collaboration among models. Experimental results demonstrate that the approach significantly outperforms existing baselines on conditional image generation tasks, effectively alleviating object omission and attribute misalignment while achieving state-of-the-art performance on metrics such as GenEval.

Abstract

The abundance of pre-trained diffusion models provides an opportunity for composition. Combining several models, however, runs the risk of one model dominating or models disagreeing with each other. Here, we propose Divide-and-Denoise, a method for coordinating multiple pre-trained diffusion models during sampling. Much like managing a specialized workforce, our method creates a fair but efficient division of labor across models. Central to our method is the notion of an allocation which defines the responsibility of each model to every region of the noisy sample. At every timestep, we then denoise by (i) updating the allocation by solving a fair division game, where we divide the sample into regions that maximize total utility under fairness constraints, and (ii) aligning the models with this allocation, where we guide each model to denoise within its assigned region. This leads to a new composite denoising process that evolves in tandem with a division process. We evaluate Divide-and-Denoise on conditional image generation. Across several quality metrics, including the GenEval benchmark, our method outperforms baselines and resolves common failures including missing objects and mismatched attributes. Experiments show that Divide-and-Denoise utilizes each model's expertise without neglecting any other model.

Problem

• diffusion models
• model composition
• fairness
• conditional image generation
• model coordination

Innovation

• diffusion model composition
• fair division
• game-theoretic allocation
• conditional image generation
• denoising coordination