Hierarchical Feature Learning for Medical Point Clouds via State Space Model

2025-04-17

AI Summary

This work addresses the emerging challenge of medical point cloud understanding by proposing the first state-space model (SSM)-based hierarchical feature learning framework. To handle point cloud irregularity and anatomical multi-scale structure, we design a multi-level downsampling scheme with KNN-based aggregation and introduce a novel dual-path serialization strategy—coordinate-order and intra-outer-order sequences. Our key contributions are: (1) the first adaptation of SSMs to medical point cloud analysis; (2) hierarchical Point SSM modules (vanilla and grouped variants) that jointly capture local geometric details and global long-range dependencies; and (3) MedPointS, the first large-scale open-source medical point cloud benchmark. Extensive experiments on MedPointS demonstrate state-of-the-art performance across anatomical classification, completion, and segmentation tasks. The code and dataset are fully open-sourced and integrated into the Flemme platform.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Medical and Biological Imaging
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Deep learning-based point cloud modeling has been widely investigated as an indispensable component of general shape analysis. Recently, transformer and state space model (SSM) have shown promising capacities in point cloud learning. However, limited research has been conducted on medical point clouds, which have great potential in disease diagnosis and treatment. This paper presents an SSM-based hierarchical feature learning framework for medical point cloud understanding. Specifically, we down-sample input into multiple levels through the farthest point sampling. At each level, we perform a series of k-nearest neighbor (KNN) queries to aggregate multi-scale structural information. To assist SSM in processing point clouds, we introduce coordinate-order and inside-out scanning strategies for efficient serialization of irregular points. Point features are calculated progressively from short neighbor sequences and long point sequences through vanilla and group Point SSM blocks, to capture both local patterns and long-range dependencies. To evaluate the proposed method, we build a large-scale medical point cloud dataset named MedPointS for anatomy classification, completion, and segmentation. Extensive experiments conducted on MedPointS demonstrate that our method achieves superior performance across all tasks. The dataset is available at https://flemme-docs.readthedocs.io/en/latest/medpoints.html. Code is merged to a public medical imaging platform: https://github.com/wlsdzyzl/flemme.

Problem

• Develops hierarchical feature learning for medical point clouds
• Introduces scanning strategies for SSM-based point cloud processing
• Builds a large-scale dataset for medical point cloud tasks

Innovation

• Hierarchical feature learning via state space model
• Coordinate-order and inside-out scanning strategies
• Vanilla and group Point SSM blocks integration