Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence

2025-03-27

AI Summary

To address the instability of 3D shape correspondence under strong non-isometric deformations—particularly its reliance on pre-alignment or high-quality initializations—this paper proposes a semantic-flow-guided differentiable mesh registration framework. Methodologically, it pioneers the use of semantic flows extracted from 2D foundation models (e.g., SAM, DINO) as geometric deformation priors to drive 3D mesh registration, enabling end-to-end joint optimization of functional maps without fine pre-alignment. Key contributions include: (1) a novel semantic-flow-guidance mechanism that bridges 2D semantics and 3D deformation; (2) effective reuse of pretrained 2D foundation models to enhance cross-domain generalization and mapping stability; and (3) overcoming performance bottlenecks of conventional functional mapping in highly non-isometric settings. Experiments demonstrate significant improvements in correspondence accuracy and robustness on non-isometric benchmarks, while supporting downstream applications such as remeshing, texture transfer, and attribute propagation.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Establishing character shape correspondence is a critical and fundamental task in computer vision and graphics, with diverse applications including re-topology, attribute transfer, and shape interpolation. Current dominant functional map methods, while effective in controlled scenarios, struggle in real situations with more complex challenges such as non-isometric shape discrepancies. In response, we revisit registration-for-correspondence methods and tap their potential for more stable shape correspondence estimation. To overcome their common issues including unstable deformations and the necessity for careful pre-alignment or high-quality initial 3D correspondences, we introduce Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence. We first re-purpose a foundation model for 2D character correspondence that ensures reliable and stable 2D mappings. Crucially, we propose a novel Semantic Flow Guided Registration approach that leverages 2D correspondence to guide mesh deformations. Our framework significantly surpasses existing methods in challenging scenarios, and brings possibilities for a wide array of real applications, as demonstrated in our results.

Problem

• Addressing unstable deformations in 3D shape correspondence
• Overcoming non-isometric shape discrepancies in real scenarios
• Eliminating need for high-quality initial 3D correspondences

Innovation

• Repurposed 2D foundation model for stable correspondence
• Semantic Flow Guided Registration for mesh deformations
• Overcomes unstable deformations and alignment issues