UnCLe: Benchmarking Continual Learning for Unsupervised Depth Completion

2024-10-23

AI Summary

Existing depth completion methods are trained on static datasets and thus struggle with non-stationary data streams, suffering severe catastrophic forgetting. This paper introduces UnCLe—the first unsupervised continual learning benchmark for depth completion—designed to simulate realistic non-stationary streams across domains (indoor/outdoor) and sensors (LiDAR/ToF). We pioneer the integration of unsupervised continual learning into depth completion, formally define “model inversion quality” as a novel metric for quantifying forgetting, and establish a standardized evaluation protocol. Our self-supervised continual training framework jointly leverages RGB and sparse depth inputs, incorporating replay, regularization, and architecture expansion strategies. Extensive experiments reveal that mainstream methods exhibit substantial forgetting and consistent cross-domain performance degradation. The UnCLe benchmark is publicly released to foster standardized research in this emerging direction.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Reasoning under Uncertainty: Sequential Decision Making
• Computer Vision: Learning & Optimization for CV

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We propose UnCLe, a standardized benchmark for Unsupervised Continual Learning of a multimodal depth estimation task: Depth completion aims to infer a dense depth map from a pair of synchronized RGB image and sparse depth map. We benchmark depth completion models under the practical scenario of unsupervised learning over continuous streams of data. Existing methods are typically trained on a static, or stationary, dataset. However, when adapting to novel non-stationary distributions, they"catastrophically forget"previously learned information. UnCLe simulates these non-stationary distributions by adapting depth completion models to sequences of datasets containing diverse scenes captured from distinct domains using different visual and range sensors. We adopt representative methods from continual learning paradigms and translate them to enable unsupervised continual learning of depth completion. We benchmark these models for indoor and outdoor and investigate the degree of catastrophic forgetting through standard quantitative metrics. Furthermore, we introduce model inversion quality as an additional measure of forgetting. We find that unsupervised continual learning of depth completion is an open problem, and we invite researchers to leverage UnCLe as a development platform.

Problem

• Benchmarking unsupervised continual learning for depth completion
• Addressing catastrophic forgetting in non-stationary data distributions
• Evaluating models on diverse indoor and outdoor scene datasets

Innovation

• Unsupervised continual learning benchmark
• Adapting to non-stationary data distributions
• Model inversion measures forgetting