Diffusion Masked Pretraining for Dynamic Point Cloud

2026-05-05

AI Summary

Existing pretraining methods for dynamic point clouds often overlook the inherent uncertainty in motion trajectories due to spatiotemporal position leakage. This work proposes DiMP, a novel framework that introduces denoising diffusion probabilistic models (DDPM) into masked pretraining for dynamic point clouds. Specifically, DiMP injects noise at the centers of masked regions and leverages visible spatiotemporal context to predict both the original positions and inter-frame displacements, thereby modeling the full distribution of motion. This approach effectively prevents positional information leakage and preserves the multimodal nature of motion uncertainty. Evaluated under a self-supervised setting, DiMP achieves significant improvements—11.21% on offline action segmentation and 13.65% in online causal inference scenarios—substantially outperforming current baselines.

Abstract

Dynamic point cloud pretraining is still dominated by masked reconstruction objectives. However, these objectives inherit two key limitations. Existing methods inject ground-truth tube centers as decoder positional embeddings, causing spatio-temporal positional leakage. Moreover, they supervise inter-frame motion with deterministic proxy targets that systematically discard distributional structure by collapsing multimodal trajectory uncertainty into conditional means. To address these limitations, we propose Diffusion Masked Pretraining (DiMP), a unified self-supervised framework for dynamic point clouds. DiMP introduces diffusion modeling into both positional inference and motion learning. It first applies forward diffusion noise only to masked tube centers, then predicts clean centers from visible spatio-temporal context. This removes positional leakage while preserving visible coordinates as clean temporal anchors. DiMP also reformulates point-wise inter-frame displacement supervision as a DDPM noise-prediction objective conditioned on decoded representations. This design drives the encoder to target the full conditional distribution of plausible motions under a variational surrogate, rather than collapsing to a single deterministic estimate. Extensive experiments demonstrate that DiMP consistently improves downstream accuracy over the backbone alone, with absolute gains of 11.21% on offline action segmentation and 13.65% under causally constrained online inference.Codes are available at https://github.com/InitalZ/DiMP.git.

Problem

• dynamic point cloud
• masked pretraining
• positional leakage
• motion uncertainty
• distributional structure

Innovation

• Diffusion Model
• Masked Pretraining
• Dynamic Point Cloud
• Self-supervised Learning
• Motion Uncertainty