Ultra: Unsupervised Cross-Task Optimization for Reliable Restoration Segmentation Collaboration under Adverse Weather

2026-08-17

AI Summary

This study addresses the challenges of ambiguous cross-task optimization directions and hallucination propagation in unsupervised domain adaptation for semantic segmentation under adverse weather conditions. We propose Ultra, a novel framework that reformulates cross-task interaction as causal effect estimation under uncertainty. By leveraging CTDN and CMIL modules to generate candidate directions and perform intervention-based filtering, Ultra achieves reliable synergy between image restoration and segmentation tasks. Extensive experiments demonstrate that the proposed framework attains state-of-the-art performance across three UDA-ASS benchmarks, outperforming existing methods in unsupervised restoration. Furthermore, Ultra successfully generalizes to collaborative object detection tasks, significantly enhancing model robustness in adverse environments.

Abstract

Unsupervised Domain Adaptation for Adverse Weather Semantic Segmentation (UDA-ASS) aims to transfer semantic knowledge from labeled normal-weather images to unlabeled adverse environments. Existing approaches implicitly assume that restoration and segmentation provide mutually beneficial guidance. However, under severe degradation and without target-domain supervision, the validity of cross-task optimization directions becomes fundamentally unidentifiable, leading to hallucination-driven error propagation. In this work, we propose a novel Unsupervised Restoration-Segmentation Collaborative Learning Framework (Ultra), which reframes cross-task interaction as direction selection under uncertainty and causal effect estimation, enabling reliable collaboration through candidate direction generation and intervention-based filtering. In detail, we propose CTDN and CMIL. The former exploits complementary visual structures and semantic information to generate candidate optimization directions and performs cooperative direction selection between restoration and segmentation. The latter reformulates cross-task information transfer from correlation-based propagation into causal effect assessment, suppressing hallucination propagation. Extensive experiments on three widely used UDA-ASS benchmarks demonstrate state-of-the-art segmentation performance. Beyond segmentation, our framework achieves better unsupervised restoration results than existing UDA-ASS restoration methods and generalizes to unsupervised restoration and object detection collaboration tasks. Code and models will be available at https://github.com/Wang-Shiqin/Ultra.

Problem

• Unsupervised Domain Adaptation
• Adverse Weather Semantic Segmentation
• Cross-Task Optimization
• Error Propagation
• Restoration-Segmentation Collaboration

Innovation

• Unsupervised Domain Adaptation
• Restoration-Segmentation Collaboration
• Causal Effect Estimation
• Direction Selection under Uncertainty
• Hallucination Suppression