🤖 AI Summary
This work challenges the prevailing notion that handcrafted local features such as SIFT should be superseded by learned methods, addressing the lack of a unified GPU-based framework for fair and modular comparison between the two paradigms. The authors introduce PySIFT—the first fully GPU-resident, bit-wise deterministic implementation of SIFT—built on CuPy and Numba, and seamlessly integrated with major deep learning frameworks via DLPack for zero-copy interoperability. Experiments demonstrate that classical features and learned matchers are complementary rather than mutually exclusive. On an RTX 3050, PySIFT outperforms OpenCV’s SIFT in both speed and accuracy: it achieves higher MMA on HPatches, reduces per-image-pair processing time on MegaDepth by 383 ms, and significantly improves cross-dataset geometric accuracy (e.g., +5.6 percentage points in AUC@10°), while ensuring consistent outputs across GPU architectures—overcoming the non-determinism inherent in cuDNN.
📝 Abstract
A widespread assumption in local feature research holds that classical handcrafted descriptors are accuracy-limited relics best replaced by learned alternatives. We show this is wrong. Through an 8-configuration ablation spanning four benchmarks (HPatches, ROxford5K, IMC Phototourism, MegaDepth), we demonstrate that classical SIFT with DSP multi-scale pooling outperforms neural descriptor and orientation replacements (HardNet, OriNet) on every accuracy metric--while running 2--18$\times$ faster--and that learned matchers (LightGlue) complement rather than supersede classical features. The conclusion reframes a decade of work: not "replace SIFT" but "compose with SIFT," classical extraction paired with learned matching only where geometric context demands it. This finding was invisible because no prior GPU SIFT kept the complete pipeline in VRAM or offered modularity for controlled classical-vs-learned ablations. We present PySIFT, the first fully GPU-resident SIFT, implemented in CuPy/Numba CUDA kernels with DLPack zero-copy handoff to downstream DL frameworks--submillisecond O(1) metadata swap regardless of keypoint count. On a laptop-grade NVIDIA RTX 3050 (4 GB VRAM), PySIFT achieves: (i) higher Mean Matching Accuracy (MMA) than OpenCV SIFT on HPatches, (ii) 383 ms faster per pair on high-resolution MegaDepth, (iii) higher geometric accuracy on cross-dataset benchmarks (+5.6 pp AUC@10${}^\circ$ on MegaDepth, more inliers on IMC Phototourism), and (iv) bitwise deterministic output--identical keypoints and descriptors across runs, with detection reproducing identically even across GPU architectures: a guarantee that learned extractors cannot match without significant performance sacrifice, and cannot achieve at all across GPU architectures due to cuDNN's architecture-dependent algorithm selection. PySIFT is open-source, requiring no C++ compilation.