Multi-modal deformable image registration using untrained neural networks

2024-11-04

arXiv.org

AI Summary

Medical image registration across diverse scenarios—including mono- and multi-modal, 2D/3D, and rigid/non-rigid transformations—typically relies on task-specific architectures, labeled data, or handcrafted similarity metrics. Method: This paper proposes a unified, model- and prior-free framework that employs a training-free CNN as an implicit regularizer for deformation field estimation. It integrates differentiable spatial transformation with unsupervised similarity measures (e.g., normalized cross-correlation, mutual information) and solves registration end-to-end via gradient-based optimization—requiring only a pair of target images. Contribution/Results: The approach eliminates dependence on annotated data, predefined similarity functions, or architectural modifications. Evaluated on heterogeneous cross-modal datasets, it achieves accuracy competitive with supervised methods while demonstrating strong generalization: no fine-tuning is needed to adapt to novel modalities or deformation types.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Image registration techniques usually assume that the images to be registered are of a certain type (e.g. single- vs. multi-modal, 2D vs. 3D, rigid vs. deformable) and there lacks a general method that can work for data under all conditions. We propose a registration method that utilizes neural networks for image representation. Our method uses untrained networks with limited representation capacity as an implicit prior to guide for a good registration. Unlike previous approaches that are specialized for specific data types, our method handles both rigid and non-rigid, as well as single- and multi-modal registration, without requiring changes to the model or objective function. We have performed a comprehensive evaluation study using a variety of datasets and demonstrated promising performance.

Problem

• Image Alignment
• Pattern Matching
• Adaptive Algorithm

Innovation

• Image Registration
• Untrained Neural Networks
• Versatile Alignment