Dual-Projection Fusion for Accurate Upright Panorama Generation in Robotic Vision

2025-11-30

AI Summary

In robotic vision, non-upright panoramic images caused by unstable platform orientation pose challenges for geometric consistency; existing IMU-based rectification methods suffer from drift and external disturbances. This paper proposes an angle-aware dual-stream network to jointly estimate camera tilt angles and generate upright panoramic images. Our method innovatively integrates equirectangular and cubic projections, and introduces a dual-projection adaptive alignment module to jointly model local geometry and global context. To enhance 360° continuity and tilt sensitivity, we incorporate high-frequency enhancement, annular padding, and channel-wise attention. Evaluated on SUN360 and M3D, our approach achieves state-of-the-art performance in both tilt estimation and panoramic reconstruction. Ablation studies validate the efficacy of each component and demonstrate synergistic gains between the two tasks.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Panoramic cameras, capable of capturing a 360-degree field of view, are crucial in robotic vision, particularly in environments with sparse features. However, non-upright panoramas due to unstable robot postures hinder downstream tasks. Traditional IMU-based correction methods suffer from drift and external disturbances, while vision-based approaches offer a promising alternative. This study presents a dual-stream angle-aware generation network that jointly estimates camera inclination angles and reconstructs upright panoramic images. The network comprises a CNN branch that extracts local geometric structures from equirectangular projections and a ViT branch that captures global contextual cues from cubemap projections. These are integrated through a dual-projection adaptive fusion module that aligns spatial features across both domains. To further enhance performance, we introduce a high-frequency enhancement block, circular padding, and channel attention mechanisms to preserve 360° continuity and improve geometric sensitivity. Experiments on the SUN360 and M3D datasets demonstrate that our method outperforms existing approaches in both inclination estimation and upright panorama generation. Ablation studies further validate the contribution of each module and highlight the synergy between the two tasks. The code and related datasets can be found at: https://github.com/YuhaoShine/DualProjectionFusion.

Problem

• Generates upright panoramas from unstable robot camera postures
• Estimates camera inclination angles without IMU drift issues
• Fuses dual projections to enhance geometric and contextual accuracy

Innovation

• Dual-stream network estimates camera inclination and reconstructs upright panoramas
• Fusion module aligns CNN and ViT features from equirectangular and cubemap projections
• High-frequency enhancement and attention mechanisms preserve 360-degree continuity