🤖 AI Summary
Shadow texture restoration is highly challenging due to the irreversible nature of shadow degradation. To address this, we propose a latent-feature-guided conditional diffusion model. First, we obtain semantically rich latent representations of shadow-free images via self-supervised learning, serving as a strong semantic prior for reconstruction. Second, we design a noise-aware feature fusion mechanism to mitigate local optima during diffusion training. Third, we incorporate multi-scale feature fusion to enhance fine-grained texture recovery. Unlike conventional weakly conditioned diffusion models that rely solely on degraded inputs, our framework pioneers semantic guidance in latent space for diffusion-based shadow removal. On the AISTD dataset, our method reduces RMSE by 13% over state-of-the-art methods; on DESOBA, it achieves an 82% improvement in instance-level shadow removal performance. These results significantly advance both accuracy and robustness in shadow region texture restoration.
📝 Abstract
Recovering textures under shadows has remained a challenging problem due to the difficulty of inferring shadow-free scenes from shadow images. In this paper, we propose the use of diffusion models as they offer a promising approach to gradually refine the details of shadow regions during the diffusion process. Our method improves this process by conditioning on a learned latent feature space that inherits the characteristics of shadow-free images, thus avoiding the limitation of conventional methods that condition on degraded images only. Additionally, we propose to alleviate potential local optima during training by fusing noise features with the diffusion network. We demonstrate the effectiveness of our approach which outperforms the previous best method by 13% in terms of RMSE on the AISTD dataset. Further, we explore instance-level shadow removal, where our model outperforms the previous best method by 82% in terms of RMSE on the DESOBA dataset.
