ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents

2025-10-24

AI Summary

This work addresses visibility changes (e.g., interior exposure upon drawer opening) and appearance distortion across multiple configurations in generative modeling of articulated 3D objects. We propose a unified generative framework jointly modeling part geometry, joint kinematics, and appearance. Methodologically, we design a structured latent space that jointly encodes geometric topology and joint attributes—including type, axis, origin, and motion range—and integrate a joint-aware Gaussian decoder for state-dependent visibility modeling and texture synthesis. Our architecture combines sparse voxel encoding, variational autoencoding, latent diffusion modeling, and joint-aware Gaussian rendering. Evaluated on PartNet-Mobility and ACD, our method achieves significant improvements in geometric consistency and cross-pose appearance fidelity, enabling high-quality, editable generation of dynamic 3D objects with physically plausible articulation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We propose ArtiLatent, a generative framework that synthesizes human-made 3D objects with fine-grained geometry, accurate articulation, and realistic appearance. Our approach jointly models part geometry and articulation dynamics by embedding sparse voxel representations and associated articulation properties, including joint type, axis, origin, range, and part category, into a unified latent space via a variational autoencoder. A latent diffusion model is then trained over this space to enable diverse yet physically plausible sampling. To reconstruct photorealistic 3D shapes, we introduce an articulation-aware Gaussian decoder that accounts for articulation-dependent visibility changes (e.g., revealing the interior of a drawer when opened). By conditioning appearance decoding on articulation state, our method assigns plausible texture features to regions that are typically occluded in static poses, significantly improving visual realism across articulation configurations. Extensive experiments on furniture-like objects from PartNet-Mobility and ACD datasets demonstrate that ArtiLatent outperforms existing approaches in geometric consistency and appearance fidelity. Our framework provides a scalable solution for articulated 3D object synthesis and manipulation.

Problem

• Generating articulated 3D objects with geometry and appearance
• Modeling part geometry and articulation dynamics in latent space
• Improving visual realism across articulation configurations

Innovation

• Generative framework embedding voxels and articulation into latent space
• Latent diffusion model enabling physically plausible object sampling
• Articulation-aware decoder improving texture realism across configurations