SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving

2026-06-15

AI Summary

This work addresses the challenge of spatial inconsistency in depth prediction caused by limited field-of-view overlap among multiple cameras. To overcome this, the authors propose SurroundNEXO, a framework that abandons conventional dense visual correspondence and instead adopts an egocentric geometric modeling approach. By integrating Ego-Ray positional encoding and sparse LiDAR metric anchors, SurroundNEXO establishes a spatially consistent depth estimation mechanism without requiring dense feature matching, enabling zero-shot generalization to arbitrary novel camera configurations. Through a staged spatio-temporal feature interaction and global fusion strategy, the method achieves significant performance gains on NuScenes, Waymo, and DDAD benchmarks—reducing monocular depth error by 33.2%, improving cross-view consistency by 10.5%, and enhancing metric reconstruction quality by 25.6%.

Abstract

Modern autonomous driving depends on accurate metric 3D understanding for perception, reconstruction, and planning, which in turn requires reliable multi-camera depth prediction. However, the outward-facing nature of vehicle-mounted surround-view camera rigs inherently limits visual overlap across views, challenging the correspondence-based assumptions that underpin conventional multi-view geometry. To bridge this gap, we present SurroundNEXO, named after the Spanish word nexo for a geometric link, a low-overlap multi-camera metric depth framework that grounds cross-view reasoning in ego-centric geometry rather than dense visual correspondences. Instead of directly enforcing early global fusion, SurroundNEXO first assigns image tokens globally comparable ego-frame viewing directions through Ego-Ray Positional Encoding, then uses sparse LiDAR measurements as metric anchors to propagate absolute scale cues, and finally expands feature interaction progressively from view-local modeling to decomposed spatio-temporal reasoning and global integration. This design enables metric-scale depth prediction with improved spatial consistency across weakly overlapping cameras. Across low-overlap autonomous driving benchmarks, including NuScenes, Waymo and DDAD, SurroundNEXO reduces single-view error by 33.2%, improves cross-view consistency by 10.5%, and enhances metric reconstruction quality by 25.6% compared with SOTA methods. It further remains robust under extremely sparse depth prompts and exhibits strong zero-shot generalization to unseen camera layouts.

Problem

• autonomous driving
• multi-camera depth prediction
• low-overlap views
• spatial consistency
• metric 3D understanding

Innovation

• ego-centric geometry
• low-overlap multi-camera
• metric depth prediction
• spatial consistency
• zero-shot generalization