Problem
Research questions and friction points this paper is trying to address.
Image Synthesis
Visual Coherence
Photorealism
Making Images Real Again: A Comprehensive Survey on Deep Image Composition
🤖 AI Summary
This work addresses the pervasive visual inconsistency—arising from mismatches in color, scale, shape, illumination, shadows, and reflections—between inserted objects and background scenes in image compositing. To this end, we propose the first holistic taxonomy and unified framework for image synthesis, encompassing core subtasks including object placement, scene fusion, color harmonization, and shadow/reflection generation. Methodologically, the framework integrates CNNs, GANs, diffusion models, and multi-scale feature alignment techniques. Our contributions are threefold: (1) a standardized benchmark suite unifying major datasets (e.g., iHarmony4, HCOCO); (2) libcom—an open-source, modular image compositing toolbox implementing over ten state-of-the-art algorithms; and (3) a paradigm shift toward systematic modeling and engineering-ready deployment in image synthesis. Extensive experiments demonstrate the framework’s generality, robustness, and practical utility across diverse compositional scenarios.
📝 Abstract
As a common image editing operation, image composition (object insertion) aims to combine the foreground from one image and another background image, resulting in a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which includes appearance inconsistency (e.g., incompatible illumination), geometry inconsistency (e.g., unreasonable size), and semantic inconsistency (e.g., mismatched semantic context). Image composition task could be decomposed into multiple sub-tasks, in which each sub-task targets at one or more issues. Specifically, object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background. Image harmonization aims to adjust the illumination statistics of foreground. Shadow (resp., reflection) generation aims to generate plausible shadow (resp., reflection) for the foreground. These sub-tasks can be executed sequentially or parallelly to acquire realistic composite images. To the best of our knowledge, there is no previous survey on image composition (object insertion). In this paper, we conduct comprehensive survey over the sub-tasks and combinatorial task of image composition (object insertion). For each one, we summarize the existing methods, available datasets, and common evaluation metrics. We have also contributed the first image composition toolbox libcom, which assembles 10+ image composition related functions.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Image Synthesis
Realism Handling
Libcom Toolbox
🔎 Similar Papers
No similar papers found.
