RobIA: Robust Instance-aware Continual Test-time Adaptation for Deep Stereo

2025-11-13

AI Summary

Stereo depth estimation in dynamic scenes faces challenges including frequent domain shifts, scarcity of dense ground-truth labels, and overreliance of existing test-time adaptation (TTA) methods on static-domain assumptions. Method: We propose an instance-aware continual test-time adaptation framework that integrates an Attend-and-Excite Mixture-of-Experts (MoE) architecture with a lightweight self-attention mechanism for input-driven dynamic expert routing; it further couples a PEFT-finetuned pseudo-labeling teacher model with Adaptive Batch Normalization (AdaptBN) to establish an adaptive pseudo-supervision loop. Contribution/Results: Our method relaxes the static target-domain assumption, enabling online, efficient, and instance-specific continual adaptation. Evaluated across multiple dynamic target domains, it significantly improves depth estimation accuracy and generalization while maintaining low computational overhead.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Stereo Depth Estimation in real-world environments poses significant challenges due to dynamic domain shifts, sparse or unreliable supervision, and the high cost of acquiring dense ground-truth labels. While recent Test-Time Adaptation (TTA) methods offer promising solutions, most rely on static target domain assumptions and input-invariant adaptation strategies, limiting their effectiveness under continual shifts. In this paper, we propose RobIA, a novel Robust, Instance-Aware framework for Continual Test-Time Adaptation (CTTA) in stereo depth estimation. RobIA integrates two key components: (1) Attend-and-Excite Mixture-of-Experts (AttEx-MoE), a parameter-efficient module that dynamically routes input to frozen experts via lightweight self-attention mechanism tailored to epipolar geometry, and (2) Robust AdaptBN Teacher, a PEFT-based teacher model that provides dense pseudo-supervision by complementing sparse handcrafted labels. This strategy enables input-specific flexibility, broad supervision coverage, improving generalization under domain shift. Extensive experiments demonstrate that RobIA achieves superior adaptation performance across dynamic target domains while maintaining computational efficiency.

Problem

• Adapting stereo depth estimation to dynamic domain shifts
• Overcoming sparse supervision with robust pseudo-label generation
• Enabling input-specific adaptation via instance-aware expert routing

Innovation

• Dynamic routing via lightweight self-attention mechanism
• Parameter-efficient experts tailored to epipolar geometry
• Teacher model providing dense pseudo-supervision for sparse labels