FRET: Feature Redundancy Elimination for Test Time Adaptation

2025-05-15

AI Summary

In test-time adaptation (TTA), distribution shifts exacerbate feature redundancy, yet existing methods overlook this issue, leading to degraded adaptation performance. To address this, this paper introduces— for the first time—feature redundancy modeling into TTA, proposing a redundancy-aware dual-paradigm framework: S-FRET, which directly optimizes redundancy via gradient-driven minimization of embedding-layer redundancy, and G-FRET, which further captures cross-sample structural relationships via graph convolutional networks and enhances discriminability through contrastive learning. Both methods operate solely on unlabeled test data, ensuring lightweight and efficient online adaptation. Extensive evaluations across diverse architectures, tasks, and datasets demonstrate state-of-the-art performance. Notably, G-FRET significantly improves robustness under label shift, achieving non-redundant, highly discriminative online feature adaptation.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Test-Time Adaptation (TTA) aims to enhance the generalization of deep learning models when faced with test data that exhibits distribution shifts from the training data. In this context, only a pre-trained model and unlabeled test data are available, making it particularly relevant for privacy-sensitive applications. In practice, we observe that feature redundancy in embeddings tends to increase as domain shifts intensify in TTA. However, existing TTA methods often overlook this redundancy, which can hinder the model's adaptability to new data. To address this issue, we introduce Feature Redundancy Elimination for Test-time Adaptation (FRET), a novel perspective for TTA. A straightforward approach (S-FRET) is to directly minimize the feature redundancy score as an optimization objective to improve adaptation. Despite its simplicity and effectiveness, S-FRET struggles with label shifts, limiting its robustness in real-world scenarios. To mitigate this limitation, we further propose Graph-based FRET (G-FRET), which integrates a Graph Convolutional Network (GCN) with contrastive learning. This design not only reduces feature redundancy but also enhances feature discriminability in both the representation and prediction layers. Extensive experiments across multiple model architectures, tasks, and datasets demonstrate the effectiveness of S-FRET and show that G-FRET achieves state-of-the-art performance. Further analysis reveals that G-FRET enables the model to extract non-redundant and highly discriminative features during inference, thereby facilitating more robust test-time adaptation.

Problem

• Eliminate feature redundancy in test-time adaptation
• Improve model adaptability to distribution shifts
• Enhance feature discriminability with GCN and contrastive learning

Innovation

• Minimizes feature redundancy for better adaptation
• Uses Graph Convolutional Network with contrastive learning
• Enhances feature discriminability in representation layers