SeeSE3: Emergence of 3D Space in Vision Features

2026-07-15

AI Summary

This work investigates whether self-supervised vision foundation models learn feature representations that align with the intrinsic structure of three-dimensional Euclidean space, even without explicit 3D supervision. To this end, we introduce a novel probing methodology based on neighborhood alignment and Poincaré adapters, along with a “latent space navigation” technique that leverages the topological and geometric relationship between the feature space and the SE(3) group to enable visual odometry and localization without explicit 3D reconstruction. Our experiments demonstrate a strong correspondence between the model’s latent subspace and 3D spatial structure, achieving accurate motion estimation and localization in static scenes using only latent features—an outcome that provides the first empirical validation of implicit 3D structural awareness in self-supervised visual representations.

Abstract

In this paper, we ask whether vision foundation models construct representations that reflect the intrinsic properties of 3D Euclidean space. Unlike previous works that probe 3D awareness of vision features by regressing image-centric quantities such as depth or normals, we investigate the relation between the structure of the space of visual features and the group of Euclidean transformations $SE(3)$. We propose a set of probes to evaluate this relation from both topological and geometric perspectives: a mutual neighborhood metric that measures the alignment between feature neighborhoods and spatial topology, and a Poincaré Adapter to test the linear accessibility of the geometry of camera motion from latent displacements in static scenes. We show that self-supervised vision models, which, in principle, have not been trained with direct 3D supervision or active agency, possess latent subspaces that are remarkably strongly correlated with three-dimensional Euclidean space, when probed correctly. Building on this insight we propose a new class of "Latent-Space Navigation" techniques that perform visual odometry and localization purely in the latent space, bypassing the need for explicit 3D reconstruction.

Problem

• 3D space
• vision foundation models
• SE(3)
• feature representation
• Euclidean geometry

Innovation

• SE(3)
• latent-space navigation
• mutual neighborhood metric
• Poincaré Adapter
• 3D awareness