MT-PCR: A Hybrid Mamba-Transformer with Spatial Serialization for Hierarchical Point Cloud Registration

2025-06-16

AI Summary

Point cloud registration (PCR) faces a trade-off between modeling capacity and computational efficiency: Transformers achieve strong long-range dependency modeling but suffer from quadratic complexity, limiting scalability to high-resolution point clouds; conversely, Mamba offers linear complexity and inherent long-range modeling yet struggles with the unordered, irregular structure of point clouds. To bridge this gap, we propose the first Mamba–Transformer hybrid framework. Our method introduces a Z-order spatial serialization scheme that explicitly preserves geometric locality and eliminates Mamba’s sensitivity to input ordering via an order-agnostic module. Further, we adopt a hierarchical architecture: a Mamba backbone efficiently encodes global context, while a lightweight Transformer performs fine-grained refinement, synergistically integrating state-space modeling (SSM) and self-attention. Evaluated on multiple benchmarks, our approach surpasses state-of-the-art methods in registration accuracy while reducing GPU memory consumption by 38% and FLOPs by 52%, achieving superior accuracy–efficiency trade-offs.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Point cloud registration (PCR) is a fundamental task in 3D computer vision and robotics. Most existing learning-based PCR methods rely on Transformers, which suffer from quadratic computational complexity. This limitation restricts the resolution of point clouds that can be processed, inevitably leading to information loss. In contrast, Mamba-a recently proposed model based on state space models (SSMs)-achieves linear computational complexity while maintaining strong long-range contextual modeling capabilities. However, directly applying Mamba to PCR tasks yields suboptimal performance due to the unordered and irregular nature of point cloud data. To address this challenge, we propose MT-PCR, the first point cloud registration framework that integrates both Mamba and Transformer modules. Specifically, we serialize point cloud features using Z-order space-filling curves to enforce spatial locality, enabling Mamba to better model the geometric structure of the input. Additionally, we remove the order indicator module commonly used in Mamba-based sequence modeling, leads to improved performance in our setting. The serialized features are then processed by an optimized Mamba encoder, followed by a Transformer refinement stage. Extensive experiments on multiple benchmarks demonstrate that MT-PCR outperforms Transformer-based and concurrent state-of-the-art methods in both accuracy and efficiency, significantly reducing while GPU memory usage and FLOPs.

Problem

• Overcomes quadratic complexity of Transformers in point cloud registration
• Integrates Mamba for linear complexity and better long-range modeling
• Serializes point clouds spatially to handle unordered, irregular data effectively

Innovation

• Hybrid Mamba-Transformer for point cloud registration
• Z-order curves serialize spatial point cloud features
• Optimized Mamba encoder with Transformer refinement