SGSoft: Learning Fused Semantic-Geometric Features for 3D Shape Correspondence via Template-Guided Soft Signals

2026-05-18

AI Summary

This work addresses the challenge of establishing dense correspondences between deformable 3D shapes exhibiting structural variations, non-isometric deformations, and topological inconsistencies. The authors propose a unified intrinsic framework that constructs a geodesic correspondence field on a canonical template, integrating pretrained semantic priors to guide multimodal dense descriptor learning. Dense correspondences are then obtained via a single forward pass through nearest-neighbor search in the learned descriptor space, eliminating the need for pre-alignment or post-optimization. The method preserves topological consistency and geometric fidelity under large pose variations, structural discrepancies, and remeshing. Experiments demonstrate state-of-the-art cross-category generalization, near real-time inference, and effective transferability of the learned descriptors to downstream tasks such as semantic segmentation and deformation transfer.

Abstract

Learning dense correspondences across deformable 3D shapes remains a long-standing challenge due to structural variability, non-isometric deformation, and inconsistent topology. Existing methods typically trade off generalization, geometric fidelity, and efficiency. We address this by proposing SGSoft, a unified intrinsic pipeline that (i) constructs a geodesic correspondence field on a canonical template, (ii) learns multimodal dense descriptors guided by pretrained semantic priors with this geodesic correspondence field supervision, (iii) retrieves dense correspondences in a single feed-forward pass via nearest-neighbor search in descriptor space. This formulation enables stable and topology-invariant supervision under large pose variation, structural differences, and remeshing. SGSoft achieves state-of-the-art inter-category generalization while offering the best accuracy-efficiency trade-off among prior methods. It also achieves near real-time inference without pre-alignment, pairwise optimization, or post-refinement. Learned descriptors can be transferred effectively to downstream tasks such as semantic segmentation and deformation transfer, establishing a scalable and deployment-ready paradigm for dense 3D correspondence.

Problem

• 3D shape correspondence
• non-isometric deformation
• structural variability
• inconsistent topology
• dense correspondences

Innovation

• 3D shape correspondence
• semantic-geometric fusion
• geodesic correspondence field
• template-guided learning
• dense descriptors