Image registration is a geometric deep learning task

2024-12-17

arXiv.org

AI Summary

To address the poor robustness, low data efficiency, and weak interpretability in deformable image registration, this paper proposes a mesh-free registration framework based on geometric deep learning. The method formulates deformation modeling within a Lagrangian reference frame—enabling multi-resolution iterative optimization without intermediate resampling for the first time. A graph neural network dynamically updates node coordinates, adaptively reconstructs neighborhoods, and incorporates geometric priors to directly learn sparse, high-dimensional deformation fields in Euclidean space. This design significantly reduces modeling errors under large deformations. Evaluated on cross-subject brain MRI and respiratory-phase lung CT registration tasks, the approach achieves state-of-the-art performance, demonstrating superior accuracy, strong generalization across domains, and physically grounded interpretability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Data-driven deformable image registration methods predominantly rely on operations that process grid-like inputs. However, applying deformable transformations to an image results in a warped space that deviates from a rigid grid structure. Consequently, data-driven approaches with sequential deformations have to apply grid resampling operations between each deformation step. While artifacts caused by resampling are negligible in high-resolution images, the resampling of sparse, high-dimensional feature grids introduces errors that affect the deformation modeling process. Taking inspiration from Lagrangian reference frames of deformation fields, our work introduces a novel paradigm for data-driven deformable image registration that utilizes geometric deep-learning principles to model deformations without grid requirements. Specifically, we model image features as a set of nodes that freely move in Euclidean space, update their coordinates under graph operations, and dynamically readjust their local neighborhoods. We employ this formulation to construct a multi-resolution deformable registration model, where deformation layers iteratively refine the overall transformation at each resolution without intermediate resampling operations on the feature grids. We investigate our method's ability to fully deformably capture large deformations across a number of medical imaging registration tasks. In particular, we apply our approach (GeoReg) to the registration of inter-subject brain MR images and inhale-exhale lung CT images, showing on par performance with the current state-of-the-art methods. We believe our contribution open up avenues of research to reduce the black-box nature of current learned registration paradigms by explicitly modeling the transformation within the architecture.

Problem

• Challenges in deformable image registration due to complex coordinate systems.
• Need for interpretable and robust deep learning architectures for registration.
• Improving performance in mono- and multi-modal brain and retinal registration.

Innovation

• Separated feature extraction and deformation modeling
• Dynamic receptive fields for spatial awareness
• Geometric deep-learning for continuous deformation modeling