Distilling Image Prototypes for Guided Test-Time Adaptation

2026-09-09

AI Summary

为解决测试时适应中的错误累积和源知识遗忘问题，本文提出DIPTTA框架，通过动态生成特征原型和校准不确定性估计来提高模型鲁棒性。

Abstract

Test-Time Adaptation (TTA) enhances the robustness of models against distribution shifts but faces two critical challenges: error accumulation from noisy pseudo-labels and catastrophic forgetting of source knowledge. Uncertainty-based approaches designed to mitigate error accumulation often yield overconfident or computationally expensive estimates, while strategies intended to prevent forgetting via prototype replay rely on static representations that easily become misaligned as the model adapts. To address these issues, this paper proposes a novel framework, Distilling Image Prototype for Guided Test-Time Adaptation (DIPTTA). The core of the proposed approach is the introduction of a Distill Image Prototype (DIP), a compact set of synthetic images that serves as a dynamic and regenerative anchor of source knowledge. This prototype enables a dynamic feature replay mechanism that continuously generates feature prototypes aligned with the current state of the model, thus effectively preventing catastrophic forgetting. Furthermore, the DIP anchors a source-calibrated uncertainty estimation method, which provides a less biased measure of sample reliability by leveraging stable source knowledge, thereby robustly suppressing error accumulation. Extensive experiments on multiple benchmarks demonstrate that DIPTTA significantly outperforms state-of-the-art methods, particularly under severe domain shifts. The source code is available at https://github.com/LiwenWang919/DIPTTA.

Problem

• Test-Time Adaptation
• error accumulation
• catastrophic forgetting

Innovation

• Test-Time Adaptation
• Distill Image Prototype
• Dynamic Feature Replay
• Catastrophic Forgetting Prevention
• Source-Calibrated Uncertainty Estimation