Sphere-VIO: Fast and Robust Visual-Inertial Odometry via Unified Spherical Representation for Heterogeneous Multi-Camera Systems

2026-06-29

AI Summary

Existing visual-inertial odometry (VIO) systems struggle to effectively support heterogeneous multi-camera setups due to the absence of a unified cross-camera representation and association mechanism, limiting the joint improvement of accuracy, robustness, and efficiency. This work proposes Sphere-VIO, which introduces the Unified Spherical Panoramic Model (USPM)—a novel formulation compatible with arbitrary standard cameras—and integrates it into a lightweight filter-based VIO framework. Sphere-VIO aligns multi-camera images via spherical bidirectional mapping, leveraging Hierarchical Omnidirectional Feature Alignment (HOFA), depth-guided semi-direct tracking, and spherical directional residuals. Efficiency is further enhanced in the error-state Kalman filter (ESKF) backend through Schur complement marginalization. Experiments demonstrate that Sphere-VIO achieves an exceptional balance among accuracy, robustness, real-time performance, and cross-camera generality on both public and custom omnidirectional datasets, making it well-suited for resource-constrained platforms.

Abstract

Multi-camera visual-inertial odometry (VIO) overcomes the inherent limitations of pure visual systems by expanding the field of view. However, existing algorithms are typically tailored for fixed camera setups and lack unified compatibility with heterogeneous multi-camera systems. Meanwhile, due to the absence of a unified cross-camera representation and association mechanism, current methods struggle to achieve a balance among robust cross-camera feature tracking, stable depth estimation, and reliable real-time performance. To address these issues, we present Sphere-VIO, a lightweight filter-based VIO framework with unified spherical representation for heterogeneous multi-camera systems. Specifically, we first propose a Unified Spherical Panorama Model (USPM) that supports all standard camera models and enables bidirectional fast mapping between multi-camera images and a shared spherical space without sequential stitching, simplifying cross-camera feature management and improving triangulation efficiency. Second, we design a parallel-accelerated depth-guided semi-direct tracking pipeline, namely Hierarchical Omnidirectional Feature Alignment (HOFA), with global spherical constraints for robust cross-camera matching, and fuse multi-camera depth observations into a standard depth filter for stable initialization. Finally, we develop a multi-camera-adapted ESKF backend that employs spherical bearing residuals and Schur complement marginalization to minimize computational overhead, enabling accurate real-time state estimation on resource-constrained devices. Extensive experiments on public benchmarks and a custom omnidirectional dataset show that Sphere-VIO achieves superior trade-offs between accuracy, robustness, efficiency, and cross-camera generality.

Problem

• visual-inertial odometry
• heterogeneous multi-camera systems
• cross-camera feature tracking
• depth estimation
• real-time performance

Innovation

• Unified Spherical Representation
• Heterogeneous Multi-Camera Systems
• Visual-Inertial Odometry
• Depth-Guided Feature Tracking
• Efficient State Estimation