Neural Collapse in Test-Time Adaptation

2025-12-11

AI Summary

Existing test-time adaptation (TTA) methods suffer performance degradation under domain shift, yet lack a theoretical explanation—rooted in misalignment between sample features and classifier weights, leading to unreliable pseudo-labels. This work first identifies and formalizes the “sample-level neural collapse” phenomenon (NC³⁺), establishing its causal link to TTA failure. We propose NCTTA, the first TTA framework explicitly designed around feature-classifier realignment: it jointly optimizes a geometric consistency objective (Euclidean distance minimization) and confidence-weighted cross-entropy loss to enforce alignment during adaptation. Evaluated on ImageNet-C, NCTTA achieves a +14.52% accuracy gain over Tent, demonstrating substantial robustness across diverse distribution shifts. Our results empirically validate that explicit alignment is a fundamental and broadly effective mechanism for TTA, offering both theoretical insight and practical advancement.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Test-Time Adaptation (TTA) enhances model robustness to out-of-distribution (OOD) data by updating the model online during inference, yet existing methods lack theoretical insights into the fundamental causes of performance degradation under domain shifts. Recently, Neural Collapse (NC) has been proposed as an emergent geometric property of deep neural networks (DNNs), providing valuable insights for TTA. In this work, we extend NC to the sample-wise level and discover a novel phenomenon termed Sample-wise Alignment Collapse (NC3+), demonstrating that a sample's feature embedding, obtained by a trained model, aligns closely with the corresponding classifier weight. Building on NC3+, we identify that the performance degradation stems from sample-wise misalignment in adaptation which exacerbates under larger distribution shifts. This indicates the necessity of realigning the feature embeddings with their corresponding classifier weights. However, the misalignment makes pseudo-labels unreliable under domain shifts. To address this challenge, we propose NCTTA, a novel feature-classifier alignment method with hybrid targets to mitigate the impact of unreliable pseudo-labels, which blends geometric proximity with predictive confidence. Extensive experiments demonstrate the effectiveness of NCTTA in enhancing robustness to domain shifts. For example, NCTTA outperforms Tent by 14.52% on ImageNet-C.

Problem

• Addresses performance degradation in Test-Time Adaptation due to domain shifts
• Identifies sample-wise misalignment between features and classifier weights as cause
• Proposes NCTTA method to realign features using hybrid targets for robustness

Innovation

• Sample-wise Alignment Collapse (NC3+) analysis
• Feature-classifier alignment method with hybrid targets
• Blends geometric proximity with predictive confidence