ETA: Energy-based Test-time Adaptation for Depth Completion

2025-08-07

AI Summary

Pretrained deep completion models suffer significant performance degradation in novel environments—e.g., varying illumination, weather, or sensor characteristics—due to covariate shift. To address this, we propose an energy-driven test-time adaptation (TTA) method. Our approach constructs a source-domain energy model to quantify prediction distribution consistency and integrates adversarial perturbations to unsupervisedly explore the underlying data manifold during inference. Crucially, it dynamically optimizes model parameters at test time by minimizing the energy score, requiring neither target-domain labels nor prior domain knowledge. Experiments on three indoor and three outdoor real-world datasets demonstrate that our method outperforms state-of-the-art approaches by +10.23% (indoor) and +6.94% (outdoor) in average completion accuracy, markedly improving cross-scenario generalization. To the best of our knowledge, this is the first work to synergistically combine energy-based modeling with adversarial perturbations for test-time adaptation in depth completion.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Adversarial Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We propose a method for test-time adaptation of pretrained depth completion models. Depth completion models, trained on some ``source'' data, often predict erroneous outputs when transferred to ``target'' data captured in novel environmental conditions due to a covariate shift. The crux of our method lies in quantifying the likelihood of depth predictions belonging to the source data distribution. The challenge is in the lack of access to out-of-distribution (target) data prior to deployment. Hence, rather than making assumptions regarding the target distribution, we utilize adversarial perturbations as a mechanism to explore the data space. This enables us to train an energy model that scores local regions of depth predictions as in- or out-of-distribution. We update the parameters of pretrained depth completion models at test time to minimize energy, effectively aligning test-time predictions to those of the source distribution. We call our method ``Energy-based Test-time Adaptation'', or ETA for short. We evaluate our method across three indoor and three outdoor datasets, where ETA improve over the previous state-of-the-art method by an average of 6.94% for outdoors and 10.23% for indoors. Project Page: https://fuzzythecat.github.io/eta.

Problem

• Adapts pretrained depth models to new environments
• Addresses covariate shift in depth prediction tasks
• Uses energy-based test-time adaptation for distribution alignment

Innovation

• Uses adversarial perturbations for data exploration
• Trains energy model to score distribution alignment
• Updates model parameters at test time