GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusion

2026-06-25

AI Summary

Existing single-image multi-view face generation methods suffer from cross-view geometric inconsistency due to the absence of explicit 3D structural constraints. This work proposes a dual-stream diffusion framework that jointly synthesizes multi-view RGB images and 3D facial geometry by introducing a viewpoint-invariant UV position map as a shared geometric representation. To enforce mutual consistency between appearance and geometry, the method incorporates a geometry-guided attention alignment loss leveraging a shared attention mechanism. Evaluated on the RenderMe-360 and NeRSemble datasets, the proposed approach significantly outperforms existing methods in both visual fidelity and cross-view geometric consistency, while also enabling more efficient 3D reconstruction.

Abstract

We present GeoFace, a geometry-constrained multi-view diffusion framework for consistent face generation from a single input. % While recent multi-view diffusion models achieve photorealistic synthesis at the per-view level, they lack an explicit mechanism to enforce a shared 3D structure across views, often leading to inconsistent geometry across viewpoints. To address this, GeoFace proposes a unified dual-stream framework for joint generation of multi-view RGB images and 3D face geometry, where the appearance and geometry streams interact through shared attention layers. To encourage the two streams to mutually constrain each other, we introduce a geometry-guided attention alignment loss that supervises the cross-attention between appearance and geometry tokens with 3D-consistent correspondences, enabling the appearance stream to correctly reference pose-invariant geometric cues for robust alignment across viewpoints. Geometry is represented as a canonical UV position map, derived from a FLAME mesh fitted to multi-view observations, serving as a view-invariant shared constraint across all generated views. Experiments on RenderMe-360 and NeRSemble demonstrate that GeoFace consistently outperforms existing methods in both visual quality and cross-view geometric consistency, facilitating more efficient 3D reconstruction.

Problem

• multi-view face generation
• geometric consistency
• 3D face reconstruction
• diffusion models
• cross-view alignment

Innovation

• geometry-constrained diffusion
• multi-view face generation
• 3D-consistent attention
• canonical UV position map
• dual-stream framework