Semi-Supervised 360 Layout Estimation with Panoramic Collaborative Perturbations

2025-03-03

AI Summary

High-quality annotated data is scarce for 360° panorama layout estimation, and existing semi-supervised methods neglect panoramic geometry and lens distortion characteristics. Method: We propose a dual-prior-driven co-perturbation semi-supervised learning framework. It is the first to jointly incorporate panoramic geometric structure priors and lens distortion-aware priors, designing a consistency-enhancing perturbation strategy tailored for unlabeled data. Additionally, we introduce boundary-focused augmentation and consistency regularization to improve layout boundary localization and model robustness. Contribution/Results: Our method achieves significant improvements over state-of-the-art approaches on three mainstream benchmarks. It substantially reduces dependency on labeled data while enhancing both layout boundary prediction accuracy and model robustness under geometric and distortion variations.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

The performance of existing supervised layout estimation methods heavily relies on the quality of data annotations. However, obtaining large-scale and high-quality datasets remains a laborious and time-consuming challenge. To solve this problem, semi-supervised approaches are introduced to relieve the demand for expensive data annotations by encouraging the consistent results of unlabeled data with different perturbations. However, existing solutions merely employ vanilla perturbations, ignoring the characteristics of panoramic layout estimation. In contrast, we propose a novel semi-supervised method named SemiLayout360, which incorporates the priors of the panoramic layout and distortion through collaborative perturbations. Specifically, we leverage the panoramic layout prior to enhance the model's focus on potential layout boundaries. Meanwhile, we introduce the panoramic distortion prior to strengthen distortion awareness. Furthermore, to prevent intense perturbations from hindering model convergence and ensure the effectiveness of prior-based perturbations, we divide and reorganize them as panoramic collaborative perturbations. Our experimental results on three mainstream benchmarks demonstrate that the proposed method offers significant advantages over existing state-of-the-art (SoTA) solutions.

Problem

• Reduces reliance on high-quality annotated data for layout estimation.
• Introduces panoramic layout and distortion priors for better accuracy.
• Ensures model convergence with effective panoramic collaborative perturbations.

Innovation

• Semi-supervised method with panoramic collaborative perturbations
• Incorporates panoramic layout and distortion priors
• Enhances model focus on layout boundaries and distortion awareness