Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

2024-12-23

AI Summary

Uniform sampling in 3D shape VAEs leads to geometric detail loss—especially sharp edges—and degraded reconstruction fidelity. Method: We propose a sharp-edge density-driven adaptive sampling strategy and a dual cross-attention mechanism: (i) a geometrically aware non-uniform point cloud sampling scheme guided by an edge saliency–derived sharp-edge density map; (ii) a lightweight dual cross-attention module that enhances joint modeling of local geometry and global structure; (iii) an edge-density-weighted shape complexity metric and Dora-bench—the first 3D VAE benchmark explicitly designed for geometric saliency. Contributions/Results: On Dora-bench, our method achieves reconstruction accuracy comparable to XCube-VAE while reducing latent code dimensionality to 1280 (an 8× reduction) and significantly improving fine-grained geometric fidelity.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent 3D content generation pipelines commonly employ Variational Autoencoders (VAEs) to encode shapes into compact latent representations for diffusion-based generation. However, the widely adopted uniform point sampling strategy in Shape VAE training often leads to a significant loss of geometric details, limiting the quality of shape reconstruction and downstream generation tasks. We present Dora-VAE, a novel approach that enhances VAE reconstruction through our proposed sharp edge sampling strategy and a dual cross-attention mechanism. By identifying and prioritizing regions with high geometric complexity during training, our method significantly improves the preservation of fine-grained shape features. Such sampling strategy and the dual attention mechanism enable the VAE to focus on crucial geometric details that are typically missed by uniform sampling approaches. To systematically evaluate VAE reconstruction quality, we additionally propose Dora-bench, a benchmark that quantifies shape complexity through the density of sharp edges, introducing a new metric focused on reconstruction accuracy at these salient geometric features. Extensive experiments on the Dora-bench demonstrate that Dora-VAE achieves comparable reconstruction quality to the state-of-the-art dense XCube-VAE while requiring a latent space at least 8$ imes$ smaller (1,280 vs.>10,000 codes). We will release our code and benchmark dataset to facilitate future research in 3D shape modeling.

Problem

• 3D Shape Variability
• Variational Autoencoders (VAEs)
• Detail Loss in Sampling

Innovation

• Dora-VAE
• 3D model reconstruction
• Dora-bench