SteeringTTA: Guiding Diffusion Trajectories for Robust Test-Time-Adaptation

2025-10-16

AI Summary

To address the degradation of classification model robustness under test-time distribution shifts, this paper proposes a diffusion-based test-time adaptation (TTA) method that requires neither model updates nor access to source data. Our approach introduces a Feynman–Kac guided mechanism, using pseudo-labels as reward signals to steer multi-particle diffusion trajectories. It further incorporates a cumulative Top-K probability metric combined with dynamic entropy scheduling to adaptively balance exploration diversity and prediction confidence. The resulting framework significantly improves cross-distortion generalization, substantially outperforming existing parameter-free TTA methods on ImageNet-C. Crucially, it operates entirely without gradients or fine-tuning during inference—enabling lightweight, robust deployment. This work establishes a novel paradigm for efficient, zero-shot test-time adaptation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Test-time adaptation (TTA) aims to correct performance degradation of deep models under distribution shifts by updating models or inputs using unlabeled test data. Input-only diffusion-based TTA methods improve robustness for classification to corruptions but rely on gradient guidance, limiting exploration and generalization across distortion types. We propose SteeringTTA, an inference-only framework that adapts Feynman-Kac steering to guide diffusion-based input adaptation for classification with rewards driven by pseudo-label. SteeringTTA maintains multiple particle trajectories, steered by a combination of cumulative top-K probabilities and an entropy schedule, to balance exploration and confidence. On ImageNet-C, SteeringTTA consistently outperforms the baseline without any model updates or source data.

Problem

• Corrects performance degradation under distribution shifts without model updates
• Guides diffusion-based input adaptation using pseudo-label rewards
• Maintains multiple particle trajectories to balance exploration and confidence

Innovation

• Guides diffusion trajectories using pseudo-label rewards
• Maintains multiple particle trajectories with top-K probabilities
• Balances exploration and confidence via entropy schedule