DiLO: Disentangled Latent Optimization for Learning Shape and Deformation in Grouped Deforming 3D Objects

2025-11-08

AI Summary

This work addresses unsupervised disentanglement of shape and deformation factors for groups of deformable 3D objects (e.g., humans, animals, faces). We propose a decoupled latent-variable optimization framework that jointly trains a generative network with two permutation-invariant PointNet encoders. Leveraging a two-stage training strategy and custom regularization, our method achieves shape–deformation separation without paired annotations. The core innovation lies in enforcing structural disentanglement directly in the latent space, enabling deformation transfer, fine-grained classification, and interpretable analysis. Extensive evaluation on multiple benchmark datasets demonstrates effectiveness: downstream task performance matches or surpasses that of complex supervised or strongly assumption-driven approaches. Our method significantly advances the practicality and generalizability of unsupervised 3D deformation modeling.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: 3D Computer Vision
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

In this work, we propose a disentangled latent optimization-based method for parameterizing grouped deforming 3D objects into shape and deformation factors in an unsupervised manner. Our approach involves the joint optimization of a generator network along with the shape and deformation factors, supported by specific regularization techniques. For efficient amortized inference of disentangled shape and deformation codes, we train two order-invariant PoinNet-based encoder networks in the second stage of our method. We demonstrate several significant downstream applications of our method, including unsupervised deformation transfer, deformation classification, and explainability analysis. Extensive experiments conducted on 3D human, animal, and facial expression datasets demonstrate that our simple approach is highly effective in these downstream tasks, comparable or superior to existing methods with much higher complexity.

Problem

• Unsupervised parameterization of 3D objects into shape and deformation factors
• Joint optimization of generator network with regularization techniques
• Amortized inference of disentangled codes for deformation transfer and classification

Innovation

• Disentangled latent optimization for shape and deformation
• Joint optimization with regularization for unsupervised learning
• PoinNet encoders for amortized inference of codes