SemFaceEdit: Semantic Face Editing on Generative Radiance Manifolds

2025-06-28

AI Summary

Existing 3D-aware GANs achieve multi-view consistency but lack fine-grained local semantic editability. To address this, we propose a semantic face editing framework grounded in generative radiance manifolds, which disentangles geometry—encoded as a semantic occupancy field—from appearance—modeled as view-consistent RGB radiance—in latent space. This enables precise, semantics-aware manipulation of facial parts (e.g., eyes, lips) while strictly preserving global geometric coherence and integrity of unedited regions. Our method employs a two-module adversarial training scheme: a geometry module jointly models semantic radiance and occupancy fields, while an appearance module synthesizes photorealistic, view-consistent RGB radiance. Extensive experiments on multiple benchmarks demonstrate substantial improvements in semantic disentanglement, editing controllability, and detail fidelity of radiance fields. The proposed approach establishes a new paradigm for high-fidelity, semantically grounded 3D face editing.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Learning with Manifolds

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Despite multiple view consistency offered by 3D-aware GAN techniques, the resulting images often lack the capacity for localized editing. In response, generative radiance manifolds emerge as an efficient approach for constrained point sampling within volumes, effectively reducing computational demands and enabling the learning of fine details. This work introduces SemFaceEdit, a novel method that streamlines the appearance and geometric editing process by generating semantic fields on generative radiance manifolds. Utilizing latent codes, our method effectively disentangles the geometry and appearance associated with different facial semantics within the generated image. In contrast to existing methods that can change the appearance of the entire radiance field, our method enables the precise editing of particular facial semantics while preserving the integrity of other regions. Our network comprises two key modules: the Geometry module, which generates semantic radiance and occupancy fields, and the Appearance module, which is responsible for predicting RGB radiance. We jointly train both modules in adversarial settings to learn semantic-aware geometry and appearance descriptors. The appearance descriptors are then conditioned on their respective semantic latent codes by the Appearance Module, facilitating disentanglement and enhanced control. Our experiments highlight SemFaceEdit's superior performance in semantic field-based editing, particularly in achieving improved radiance field disentanglement.

Problem

• Enables localized semantic editing of 3D-aware GAN-generated faces
• Disentangles geometry and appearance for precise facial attribute control
• Reduces computational costs while learning fine facial details

Innovation

• Generates semantic fields on radiance manifolds
• Disentangles geometry and appearance via latent codes
• Enables precise facial editing preserving other regions