PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation

2025-06-03

AI Summary

Continual Test-Time Adaptation (CTTA) often neglects the prior knowledge embedded in pretrained model weights, particularly their geometric structure. Method: We observe that pairwise angular relationships among weight vectors remain highly stable under cross-domain perturbations, encoding domain-invariant semantics. Leveraging this insight, we propose *angular invariance* as a core constraint: weight vectors are decoupled into magnitude and direction components; the latter is transformed via a learnable Householder orthogonal matrix to preserve inter-vector angles globally, while magnitudes and orthogonal parameters are jointly optimized online. Contribution/Results: This is the first work to systematically exploit and leverage the geometric invariance of pretrained weights—without requiring source data or additional annotations. Our method achieves state-of-the-art performance across four mainstream CTTA benchmarks, demonstrating that preserving angular structure effectively models continual domain shifts and enhances generalization.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Continual Test-Time Adaptation (CTTA) aims to online adapt a pre-trained model to changing environments during inference. Most existing methods focus on exploiting target data, while overlooking another crucial source of information, the pre-trained weights, which encode underutilized domain-invariant priors. This paper takes the geometric attributes of pre-trained weights as a starting point, systematically analyzing three key components: magnitude, absolute angle, and pairwise angular structure. We find that the pairwise angular structure remains stable across diverse corrupted domains and encodes domain-invariant semantic information, suggesting it should be preserved during adaptation. Based on this insight, we propose PAID (Pairwise Angular-Invariant Decomposition), a prior-driven CTTA method that decomposes weight into magnitude and direction, and introduces a learnable orthogonal matrix via Householder reflections to globally rotate direction while preserving the pairwise angular structure. During adaptation, only the magnitudes and the orthogonal matrices are updated. PAID achieves consistent improvements over recent SOTA methods on four widely used CTTA benchmarks, demonstrating that preserving pairwise angular structure offers a simple yet effective principle for CTTA.

Problem

• Exploiting pre-trained weights for domain-invariant priors in CTTA
• Preserving pairwise angular structure during model adaptation
• Improving continual test-time adaptation via geometric weight decomposition

Innovation

• Preserves pairwise angular structure during adaptation
• Decomposes weight into magnitude and direction
• Uses learnable orthogonal matrix via Householder reflections