DM3D: Deformable Mamba via Offset-Guided Gaussian Sequencing for Point Cloud Understanding

2025-12-03

AI Summary

To resolve the fundamental conflict between point cloud disorder and the input-order dependency of State Space Models (SSMs), this paper proposes a deformable Mamba architecture. Methodologically, it introduces deformable scanning for point cloud serialization—first incorporating offset-guided Gaussian KNN resampling and differentiable Gaussian reordering to jointly achieve local geometric adaptivity and global structural awareness. A triple-path frequency fusion module is further designed to enhance spectral-domain modeling. The entire serialization process is end-to-end optimized, significantly improving SSMs’ capability to jointly capture long-range dependencies and fine-grained local structures in point clouds. Extensive experiments demonstrate state-of-the-art performance on classification, few-shot learning, and part segmentation tasks. Results validate that structure-adaptive serialization is pivotal for unlocking the full modeling potential of SSMs on point cloud data.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

State Space Models (SSMs) demonstrate significant potential for long-sequence modeling, but their reliance on input order conflicts with the irregular nature of point clouds. Existing approaches often rely on predefined serialization strategies, which cannot adjust based on diverse geometric structures. To overcome this limitation, we propose extbf{DM3D}, a deformable Mamba architecture for point cloud understanding. Specifically, DM3D introduces an offset-guided Gaussian sequencing mechanism that unifies local resampling and global reordering within a deformable scan. The Gaussian-based KNN Resampling (GKR) enhances structural awareness by adaptively reorganizing neighboring points, while the Gaussian-based Differentiable Reordering (GDR) enables end-to-end optimization of serialization order. Furthermore, a Tri-Path Frequency Fusion module enhances feature complementarity and reduces aliasing. Together, these components enable structure-adaptive serialization of point clouds. Extensive experiments on benchmark datasets show that DM3D achieves state-of-the-art performance in classification, few-shot learning, and part segmentation, demonstrating that adaptive serialization effectively unlocks the potential of SSMs for point cloud understanding.

Problem

• Adaptively serializing irregular point clouds for SSMs
• Unifying local resampling and global reordering via deformable scans
• Enabling structure-aware point cloud understanding with Mamba models

Innovation

• Offset-guided Gaussian sequencing for point cloud serialization
• Gaussian-based KNN resampling and differentiable reordering
• Tri-Path Frequency Fusion module for feature enhancement