Test-time Correlation Alignment

2025-05-01

AI Summary

To address the challenge of test-time distribution shift without access to source-domain data or labels, this paper proposes LinearTCA—a backpropagation-free test-time adaptation (TTA) method—and its enhanced variant, LinearTCA+. Differing from prevailing instance-level alignment strategies, LinearTCA is the first TTA approach to incorporate covariance alignment (CORAL), with theoretical guarantees on effectiveness and robustness against domain forgetting. It employs linear transformation optimization, high-confidence sample selection, and gradient-free covariance matching to achieve superior efficiency and architectural compatibility. On OfficeHome, LinearTCA+ improves accuracy by 5.88% over strong baselines, reduces GPU memory consumption to just 4% of the best existing method, and incurs only 0.6% additional inference latency. These advances collectively establish new state-of-the-art performance across key TTA metrics.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Deep neural networks often experience performance drops due to distribution shifts between training and test data. Although domain adaptation offers a solution, privacy concerns restrict access to training data in many real-world scenarios. This restriction has spurred interest in Test-Time Adaptation (TTA), which adapts models using only unlabeled test data. However, current TTA methods still face practical challenges: (1) a primary focus on instance-wise alignment, overlooking CORrelation ALignment (CORAL) due to missing source correlations; (2) complex backpropagation operations for model updating, resulting in overhead computation and (3) domain forgetting. To address these challenges, we provide a theoretical analysis to investigate the feasibility of Test-time Correlation Alignment (TCA), demonstrating that correlation alignment between high-certainty instances and test instances can enhance test performances with a theoretical guarantee. Based on this, we propose two simple yet effective algorithms: LinearTCA and LinearTCA+. LinearTCA applies a simple linear transformation to achieve both instance and correlation alignment without additional model updates, while LinearTCA+ serves as a plug-and-play module that can easily boost existing TTA methods. Extensive experiments validate our theoretical insights and show that TCA methods significantly outperforms baselines across various tasks, benchmarks and backbones. Notably, LinearTCA improves adaptation accuracy by 5.88% on OfficeHome dataset, while using only 4% maximum GPU memory usage and 0.6% computation time compared to the best baseline TTA method.

Problem

• Address performance drops from training-test distribution shifts
• Enable correlation alignment without source data access
• Reduce computation overhead in test-time adaptation

Innovation

• Test-time Correlation Alignment (TCA) for domain adaptation
• LinearTCA achieves alignment without model updates
• LinearTCA+ boosts existing TTA methods plug-and-play