RealKeyMorph: Keypoints in Real-world Coordinates for Resolution-agnostic Image Registration

2025-06-12

AI Summary

Medical image registration in practice often encounters multi-resolution challenges—such as varying pixel spacing and slice thickness—where existing deep learning methods typically resample inputs to a fixed resolution, introducing interpolation artifacts. To address this, we propose the first end-to-end resolution-agnostic registration framework: it eliminates input resampling entirely and directly regresses detected anatomical landmarks into the scanner’s physical coordinate system. Building upon the KeyMorph architecture, our method integrates device-provided affine matrices to align coordinate spaces and jointly leverages keypoint detection and closed-form matching to analytically solve for the deformation field. We validate the framework on abdominal MR 2D orthogonal slices and multi-resolution 3D brain datasets, achieving an average Dice score improvement of 3.2% over baseline methods. Moreover, the approach demonstrates robustness across scanners and imaging protocols.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Many real-world settings require registration of a pair of medical images that differ in spatial resolution, which may arise from differences in image acquisition parameters like pixel spacing, slice thickness, and field-of-view. However, all previous machine learning-based registration techniques resample images onto a fixed resolution. This is suboptimal because resampling can introduce artifacts due to interpolation. To address this, we present RealKeyMorph (RKM), a resolution-agnostic method for image registration. RKM is an extension of KeyMorph, a registration framework which works by training a network to learn corresponding keypoints for a given pair of images, after which a closed-form keypoint matching step is used to derive the transformation that aligns them. To avoid resampling and enable operating on the raw data, RKM outputs keypoints in real-world coordinates of the scanner. To do this, we leverage the affine matrix produced by the scanner (e.g., MRI machine) that encodes the mapping from voxel coordinates to real world coordinates. By transforming keypoints into real-world space and integrating this into the training process, RKM effectively enables the extracted keypoints to be resolution-agnostic. In our experiments, we demonstrate the advantages of RKM on the registration task for orthogonal 2D stacks of abdominal MRIs, as well as 3D volumes with varying resolutions in brain datasets.

Problem

• Registration of medical images with different resolutions
• Avoiding artifacts from resampling during image registration
• Enabling resolution-agnostic keypoints in real-world coordinates

Innovation

• Resolution-agnostic image registration method
• Keypoints in real-world scanner coordinates
• Affine matrix integration for voxel mapping