SGMatch: Semantic-Guided Non-Rigid Shape Matching with Flow Regularization

2026-03-13

AI Summary

This work addresses the challenge of establishing robust point-to-point correspondences across non-rigid 3D shapes under anisometric deformations and topological noise. To this end, the authors propose a semantic-guided non-rigid shape matching framework that integrates semantic features extracted by vision foundation models with geometric descriptors. A novel semantic-guided local cross-attention module is introduced to effectively fuse these modalities, and—remarkably for the first time—conditional flow matching regularization is employed to supervise the correspondence velocity field, thereby enhancing spatial smoothness and structural consistency. Furthermore, spectral basis projection refinement is incorporated to improve matching accuracy. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method significantly outperforms state-of-the-art approaches, particularly excelling in scenarios involving severe anisometric deformations and topological perturbations.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Establishing accurate point-to-point correspondences between non-rigid 3D shapes remains a critical challenge, particularly under non-isometric deformations and topological noise. Existing functional map pipelines suffer from ambiguities that geometric descriptors alone cannot resolve, and spatial inconsistencies inherent in the projection of truncated spectral bases to dense pointwise correspondences. In this paper, we introduce SGMatch, a learning-based framework for semantic-guided non-rigid shape matching. Specifically, we design a Semantic-Guided Local Cross-Attention module that integrates semantic features from vision foundation models into geometric descriptors while preserving local structural continuity. Furthermore, we introduce a regularization objective based on conditional flow matching, which supervises a time-varying velocity field to encourage spatial smoothness of the recovered correspondences. Experimental results on multiple benchmarks demonstrate that SGMatch achieves competitive performance across near-isometric settings and consistent improvements under non-isometric deformations and topological noise.

Problem

• non-rigid shape matching
• point-to-point correspondence
• non-isometric deformations
• topological noise
• 3D shape correspondence

Innovation

• Semantic-Guided Matching
• Non-Rigid Shape Correspondence
• Conditional Flow Matching
• Local Cross-Attention
• Functional Maps