DINOcular: Self-Supervised Visuospatial Representations

2026-08-27

AI Summary

本文提出一种自监督框架DINOcular，通过融合RGB-D观测中的几何先验和视觉特征，有效提升3D感知能力，同时保持语义分割性能。

Abstract

We introduce a self-supervised framework for learning joint visuospatial representations from RGB-D observations. While modern vision foundation models are trained almost exclusively on RGB images, many embodied systems have access to explicit depth sensing, which provides geometric information that monocular inputs cannot recover. Our method integrates depth-derived geometric priors with a visual backbone through inter-patch and intra-patch fusion, enabling the model to encode both appearance and spatial structure efficiently. The resulting representation shows promising improvements on 3D awareness while preserving semantic transfer: it outperforms prior methods of comparable scale on multiple 3D geometry benchmarks, and remains competitive when probed for standard RGB-D semantic segmentation tasks.

Problem

• self-supervised
• visuospatial representations
• RGB-D observations
• geometric priors
• semantic transfer

Innovation

• self-supervised framework
• visuospatial representations
• RGB-D observations
• geometric priors
• inter-patch and intra-patch fusion