Unsupervised MRI-US Multimodal Image Registration with Multilevel Correlation Pyramidal Optimization

2026-02-06

AI Summary

This work addresses the challenge of registering multimodal medical images—such as preoperative MRI and intraoperative ultrasound—which is hindered by inter-modality discrepancies and tissue deformations. The authors propose an unsupervised multimodal registration method that employs a modality-invariant neighborhood descriptor for feature extraction, integrates a multi-scale correlation pyramid for dense matching, and introduces a weight-balanced coupled convex optimization strategy to jointly enforce global consistency and preserve local anatomical details in the displacement field. Evaluated on the ReMIND2Reg task of Learn2Reg 2025, the method achieved first place in both validation and test phases. On the Resect dataset, it attained a mean target registration error (TRE) of 1.798 mm, demonstrating exceptional accuracy and strong clinical applicability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Surgical navigation based on multimodal image registration has played a significant role in providing intraoperative guidance to surgeons by showing the relative position of the target area to critical anatomical structures during surgery. However, due to the differences between multimodal images and intraoperative image deformation caused by tissue displacement and removal during the surgery, effective registration of preoperative and intraoperative multimodal images faces significant challenges. To address the multimodal image registration challenges in Learn2Reg 2025, an unsupervised multimodal medical image registration method based on multilevel correlation pyramidal optimization (MCPO) is designed to solve these problems. First, the features of each modality are extracted based on the modality independent neighborhood descriptor, and the multimodal images is mapped to the feature space. Second, a multilevel pyramidal fusion optimization mechanism is designed to achieve global optimization and local detail complementation of the displacement field through dense correlation analysis and weight-balanced coupled convex optimization for input features at different scales. Our method focuses on the ReMIND2Reg task in Learn2Reg 2025. Based on the results, our method achieved the first place in the validation phase and test phase of ReMIND2Reg. The MCPO is also validated on the Resect dataset, achieving an average TRE of 1.798 mm. This demonstrates the broad applicability of our method in preoperative-to-intraoperative image registration. The code is avaliable at https://github.com/wjiazheng/MCPO.

Problem

• multimodal image registration
• unsupervised learning
• intraoperative deformation
• MRI-US registration
• surgical navigation

Innovation

• unsupervised registration
• multimodal image registration
• pyramidal optimization
• MIND descriptor
• convex optimization