KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models

2025-12-03

AI Summary

This work addresses unsupervised learning of structured 3D keypoints from point clouds, aiming to obtain compact, interpretable, and spatially consistent geometric representations for shape completion. We propose the first framework that couples unsupervised keypoint learning with latent-space diffusion modeling: a lightweight keypoint encoder is jointly optimized with an Elucidated Diffusion Model (EDM) in an unconditional generation setting, enabling concurrent learning of keypoint distributions and shape reconstruction. Key innovations include smooth interpolation in keypoint space and cross-instance structural alignment, substantially improving geometric deformation modeling. Evaluated on multi-category ShapeNet, our method achieves a 6-percentage-point improvement in keypoint consistency over state-of-the-art methods and enables high-fidelity, controllable shape completion.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Understanding and representing the structure of 3D objects in an unsupervised manner remains a core challenge in computer vision and graphics. Most existing unsupervised keypoint methods are not designed for unconditional generative settings, restricting their use in modern 3D generative pipelines; our formulation explicitly bridges this gap. We present an unsupervised framework for learning spatially structured 3D keypoints from point cloud data. These keypoints serve as a compact and interpretable representation that conditions an Elucidated Diffusion Model (EDM) to reconstruct the full shape. The learned keypoints exhibit repeatable spatial structure across object instances and support smooth interpolation in keypoint space, indicating that they capture geometric variation. Our method achieves strong performance across diverse object categories, yielding a 6 percentage-point improvement in keypoint consistency compared to prior approaches.

Problem

• Unsupervised learning of 3D object structure from point clouds
• Bridging unsupervised keypoints with generative diffusion models
• Learning interpretable keypoints for shape reconstruction and interpolation

Innovation

• Unsupervised 3D keypoint learning from point clouds
• Keypoints condition a diffusion model for shape reconstruction
• Keypoints capture geometric variation with consistent structure