Structural Energy Guidance for View-Consistent Text-to-3D Generation

2026-05-19

AI Summary

This work addresses the Janus problem in text-to-3D generation, where 2D diffusion priors often introduce view-dependent biases that compromise multi-view geometric consistency. To mitigate this issue, the authors propose Structure Energy-Guided Sampling (SEGS), a novel approach that incorporates structural energy directly into the diffusion sampling process without requiring additional training. Specifically, SEGS constructs a structure energy within the PCA subspace of U-Net features and injects its gradient into the denoising steps, enabling a plug-and-play enhancement of multi-view consistency. The method seamlessly integrates with mainstream frameworks such as SDS and VSD, consistently reducing Janus rates by approximately 10% across DreamFusion, Magic3D, and LucidDreamer, while significantly improving View-CS scores and preserving visual fidelity.

Abstract

Text-to-3D generation based on diffusion models often suffers from the Janus problem, leading to inconsistent geometry across viewpoints. This work identifies viewpoint bias in 2D diffusion priors as the main cause and proposes Structural Energy-Guided Sampling (SEGS), a training-free and plug-and-play framework to improve multi-view consistency. SEGS constructs a structural energy in the PCA subspace of U-Net features and injects its gradient into the denoising process. It can be easily integrated into SDS/VSD pipelines without retraining. Experiments show that SEGS reduces the Janus Rate by about 10% on average and improves View-CS scores across multiple baselines, including DreamFusion, Magic3D, and LucidDreamer. This method effectively alleviates viewpoint artifacts while preserving appearance fidelity, providing a flexible solution for high-quality text-to-3D content generation.

Problem

• Text-to-3D
• Janus problem
• view consistency
• diffusion models
• multi-view inconsistency

Innovation

• Structural Energy Guidance
• View-Consistent Text-to-3D
• Diffusion Models
• Janus Problem
• Training-Free Framework