A Streamlined Attention-Based Network for Descriptor Extraction

2026-01-19

AI Summary

This work proposes SANDesc, a lightweight attention-driven keypoint descriptor network that significantly enhances image matching performance without modifying the underlying detector. Built upon an improved residual U-Net architecture, SANDesc integrates Convolutional Block Attention Modules (CBAM) and employs a refined triplet loss combined with a curriculum learning-inspired hard negative mining strategy. The model is highly efficient, containing only 2.4 million parameters. It consistently outperforms existing descriptors on standard benchmarks including HPatches, MegaDepth-1500, and IMC 2021, and demonstrates substantial performance gains on a newly introduced high-resolution urban 4K dataset.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We introduce SANDesc, a Streamlined Attention-Based Network for Descriptor extraction that aims to improve on existing architectures for keypoint description. Our descriptor network learns to compute descriptors that improve matching without modifying the underlying keypoint detector. We employ a revised U-Net-like architecture enhanced with Convolutional Block Attention Modules and residual paths, enabling effective local representation while maintaining computational efficiency. We refer to the building blocks of our model as Residual U-Net Blocks with Attention. The model is trained using a modified triplet loss in combination with a curriculum learning-inspired hard negative mining strategy, which improves training stability. Extensive experiments on HPatches, MegaDepth-1500, and the Image Matching Challenge 2021 show that training SANDesc on top of existing keypoint detectors leads to improved results on multiple matching tasks compared to the original keypoint descriptors. At the same time, SANDesc has a model complexity of just 2.4 million parameters. As a further contribution, we introduce a new urban dataset featuring 4K images and pre-calibrated intrinsics, designed to evaluate feature extractors. On this benchmark, SANDesc achieves substantial performance gains over the existing descriptors while operating with limited computational resources.

Problem

• keypoint description
• descriptor extraction
• image matching
• feature matching
• computational efficiency

Innovation

• attention mechanism
• lightweight descriptor
• residual U-Net
• hard negative mining
• feature matching