Random Registers for Cross-Domain Few-Shot Learning

2025-06-03

AI Summary

Cross-domain few-shot learning (CDFSL) suffers from severe distributional shift between source and target domains, leading to poor transferability. We observe that learnable prompts—commonly used in Vision Transformers (ViTs)—tend to overfit domain-specific noise during source-domain training, resulting in sharp loss landscapes and degraded generalization to the target domain. In contrast, replacing them with fixed random registers acts as an implicit attention perturbation mechanism, encouraging sharpness-aware optimization and guiding the model toward flatter, more transferable minima. Building on this insight, we propose the Random Register Framework, augmented with a semantic region enhancement strategy to further improve robustness. Our method achieves state-of-the-art performance across four major CDFSL benchmarks, significantly boosting cross-domain few-shot classification accuracy. Code and pretrained models are publicly available.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Cross-domain few-shot learning (CDFSL) aims to transfer knowledge from a data-sufficient source domain to data-scarce target domains. Although Vision Transformer (ViT) has shown superior capability in many vision tasks, its transferability against huge domain gaps in CDFSL is still under-explored. In this paper, we find an intriguing phenomenon: during the source-domain training, prompt tuning, as a common way to train ViT, could be harmful for the generalization of ViT in target domains, but setting them to random noises (i.e., random registers) could consistently improve target-domain performance. We then delve into this phenomenon for an interpretation. We find that learnable prompts capture domain information during the training on the source dataset, which views irrelevant visual patterns as vital cues for recognition. This can be viewed as a kind of overfitting and increases the sharpness of the loss landscapes. In contrast, random registers are essentially a novel way of perturbing attention for the sharpness-aware minimization, which helps the model find a flattened minimum in loss landscapes, increasing the transferability. Based on this phenomenon and interpretation, we further propose a simple but effective approach for CDFSL to enhance the perturbation on attention maps by adding random registers on the semantic regions of image tokens, improving the effectiveness and efficiency of random registers. Extensive experiments on four benchmarks validate our rationale and state-of-the-art performance. Codes and models are available at https://github.com/shuaiyi308/REAP.

Problem

• Enhancing cross-domain few-shot learning transferability with Vision Transformers
• Addressing overfitting in prompt tuning for improved target-domain performance
• Optimizing random registers to flatten loss landscapes for better generalization

Innovation

• Random registers improve target-domain performance
• Perturbing attention for sharpness-aware minimization
• Adding random registers on semantic regions