🤖 AI Summary
This work addresses the poor generalization of Neural Radiance Fields (NeRF) in few-shot novel-view synthesis—specifically, from only two input views. To enhance geometric consistency, we propose an explicit geometric prior modeling framework. Our method introduces three key innovations: (1) a cosine-similarity-driven 2D–3D–2D cross-view point correspondence mechanism, serving as an explicit geometric guidance signal; (2) a Transformer-based cross-attention module to model robust multi-view feature interactions; and (3) joint optimization of color and density prediction via multi-view feature projection integrated with volumetric rendering. Evaluated under diverse generalization settings—including unseen scenes, poses, and domains—our approach achieves state-of-the-art performance. Ablation studies reveal that the learned feature similarity strongly correlates with volumetric density, significantly improving both visual quality and geometric fidelity in extremely sparse-view settings.
📝 Abstract
We present a new generalizable NeRF method that is able to directly generalize to new unseen scenarios and perform novel view synthesis with as few as two source views. The key to our approach lies in the explicitly modeled correspondence matching information, so as to provide the geometry prior to the prediction of NeRF color and density for volume rendering. The explicit correspondence matching is quantified with the cosine similarity between image features sampled at the 2D projections of a 3D point on different views, which is able to provide reliable cues about the surface geometry. Unlike previous methods where image features are extracted independently for each view, we consider modeling the cross-view interactions via Transformer cross-attention, which greatly improves the feature matching quality. Our method achieves state-of-the-art results on different evaluation settings, with the experiments showing a strong correlation between our learned cosine feature similarity and volume density, demonstrating the effectiveness and superiority of our proposed method. Code and pretrained weights are at https://github.com/donydchen/matchnerf.
