Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate

2026-06-22

AI Summary

This work systematically investigates the fundamental reasons why conditional diffusion models fail to extrapolate to unseen combinations in compositional generation tasks. Focusing on models trained only on a subset of possible condition combinations, the authors employ theoretical generalization analysis alongside experiments on both synthetic and real-world data to demonstrate how score estimation errors lead to catastrophic failure in out-of-distribution compositional settings. The study critically challenges the efficacy of existing inference-time correction methods, showing that even advanced techniques such as Feynman–Kac-based refinements suffer severe performance degradation when the target distribution deviates substantially from the source training distribution. These findings underscore the urgent need for novel modeling paradigms capable of achieving reliable compositional generalization.

Abstract

The task of compositional generation involves using a conditional generative model, trained only on a subset of the possible conditions, to produce samples from compositionally-defined target distributions such as a geometric combination of the source distributions. In this work, we argue that this task is often infeasible for vanilla conditional diffusion models: we conjecture that no inference-time technique can efficiently produce samples from the target distribution in certain well-motivated settings. This idea is supported by theory-guided generalization arguments and carefully-designed experiments on both synthetic and realistic data. In particular, while recent methods such as Feynman-Kac correction reduce inference-time approximation error, our results show that score estimation error has a more catastrophic effect on performance when the target distribution is out-of-distribution with respect to the sources, highlighting the need for a different approach to this task.

Problem

• compositional generation
• diffusion models
• out-of-distribution
• extrapolation
• score estimation

Innovation

• compositional generation
• diffusion models
• score estimation error
• out-of-distribution generalization
• Feynman-Kac correction