Problem
Research questions and friction points this paper is trying to address.
multimodal scenarios
different sensing processes
cross-modal description
CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space
🤖 AI Summary
本文提出CrossFeat框架，通过在特征描述子空间学习转换函数，使单模态描述子能够跨模态工作，解决多模态图像匹配问题。
📝 Abstract
Most advances in keypoint descriptions address monomodal settings, where image variations arise from viewpoint, illumination, or contrast changes. Multimodal scenarios involve images produced by fundamentally different sensing processes, such as multispectral imaging, RGB-depth, satellite imagery, or medical imaging, causing the same structures to appear differently. A common solution to cross-modal description is to train descriptors for each modality pair, which requires retraining whenever the modalities change, or to train large models, which incur a significant increase in runtime. Instead, we propose CrossFeat, a framework that enables an existing monomodal descriptor to operate across modalities. Our method learns a crossing function in descriptor space that maps features from one modality to a representation compatible with another. To preserve the structural information captured by the original descriptor, CrossFeat introduces a geometry-appearance disentanglement such that only appearance is altered while the geometric properties are preserved. Experiments across multiple domains and datasets demonstrate improved performance in multimodal matching.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Cross-modal Description
Feature Descriptor Space
Geometry-Appearance Disentanglement
Multimodal Matching
🔎 Similar Papers
No similar papers found.
