Reliability-Guided Adaptive Ensembling for Robust Test-Time Adaptation

2026-06-21

AI Summary

This work addresses the poor robustness and unstable online updates of test-time adaptation (TTA) under adversarially corrupted test streams by proposing SAFER, a training-free robust TTA framework. SAFER constructs stable predictions through reliability-guided stochastic augmentation and correlation-weighted pooling, and introduces an adaptive ensembling strategy based on feature divergence to simultaneously enhance adversarial robustness and preserve performance on clean samples. As the first systematic study of robust TTA in the adversarial streaming setting, SAFER significantly improves the robustness of diverse TTA methods against PGD attacks on PACS, VLCS, and OfficeHome datasets while maintaining competitive accuracy on clean data.

Abstract

Test-time adaptation (TTA) can mitigate domain shift without source data, but it is highly brittle under adversarially contaminated test streams, where corrupted inputs also destabilize online updates. We study robust test-time adaptation (RTTA) in the adversarial-stream setting, which remains comparatively underexplored relative to standard TTA, and propose SAFER (Stochastic Augmentation Framework for Enhanced Robustness), a training-free reliability-guided augmentation wrapper for RTTA. SAFER preserves the wrapped TTA objective while replacing brittle single-view predictions with a reliability-guided pooled predictor. For each test sample, SAFER generates stochastic augmentations and aggregates their predictions through correlation-weighted pooling with outlier detection. We further study an adaptive-mixing extension that improves clean-performance retention by adjusting original-versus-augmentation weighting using feature disagreement signals. We evaluate on PACS, VLCS, and OfficeHome under PGD attacks at various attack rates. Across benchmarks, SAFER improves resilience of TTA methods to adversarial attacks while maintaining competitive clean performance.

Problem

• test-time adaptation
• adversarial contamination
• domain shift
• robustness
• online updates

Innovation

• Test-Time Adaptation
• Adversarial Robustness
• Stochastic Augmentation
• Reliability-Guided Pooling
• Domain Shift