AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation

2026-06-30

AI Summary

Existing image synthesis methods often struggle to simultaneously achieve 3D structural consistency and high photorealism, frequently introducing over-constrained artifacts due to excessive reliance on visual cues. This work proposes an adaptive conditioning framework based on diffusion models that dynamically modulates ControlNet guidance strength through a self-supervised mechanism, thereby mitigating over-constraint while preserving generative expressiveness. Additionally, a multi-agent vision-language model is introduced to produce semantically rich textual prompts aligned with 3D geometry. The proposed approach significantly enhances both visual fidelity and 3D consistency of synthesized images, enabling the generation of high-quality, scalable datasets with precise 2D/3D annotations, which demonstrate superior utility in downstream tasks.

Abstract

Synthetic data generation has emerged as a powerful tool for improving data scalability in computer vision. Recent diffusion-based pipelines have demonstrated strong photorealism. However, how to enforce precise 3D structure and pose consistency in generated images remains challenging. Existing methods leverage visual prompts such as edge maps to guide diffusion models, but often suffer from over-conditioning artifacts that degrade image realism and limit dataset quality. In this paper, we present a diffusion-based image generation framework that enforces 3D structural alignment while preserving photorealism through adaptive conditioning. Our framework, Adaptive Conditioning for 3D-Aware Synthetic Data Generation (AC3S), introduces a self-supervised visual prompt modulator that dynamically adjusts the strength of ControlNet conditioning, preventing over-conditioning and enabling the diffusion model to retain its generative expressiveness. To further enhance diversity and semantic consistency, we develop a multi-agent vision language model framework that composes detailed and 3D-aware prompts aligned with the underlying geometric structure. Together, these components enable the scalable generation of high-quality synthetic datasets with accurate 2D and 3D annotations. Extensive experiments demonstrate that our method significantly improves image quality and downstream utility.

Problem

• 3D-aware generation
• synthetic data
• pose consistency
• over-conditioning
• photorealism

Innovation

• adaptive conditioning
• 3D-aware generation
• diffusion models
• ControlNet
• synthetic data