SelfOccFlow: Towards End-to-End Self-Supervised 3D Occupancy Flow Prediction

2026-02-27

IEEE Robotics and Automation Letters

AI Summary

This work proposes the first end-to-end fully self-supervised method for joint 3D occupancy and motion estimation in autonomous driving, addressing the reliance on costly human annotations or external supervision. By decoupling static and dynamic signed distance fields and incorporating temporal feature aggregation with a cosine similarity constraint on features, the model implicitly learns scene dynamics without any manual labels. The approach is validated on SemanticKITTI, KITTI-MOT, and nuScenes datasets, demonstrating significant reduction in dependence on annotated data. Furthermore, it introduces a strong self-supervised optical flow cue derived from feature similarity, advancing the state of self-supervised modeling of dynamic 3D scenes.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely on expensive 3D occupancy and flow annotations, velocity labels from bounding boxes, or pretrained optical flow models. We propose a self-supervised method for 3D occupancy flow estimation that eliminates the need for human-produced annotations or external flow supervision. Our method disentangles the scene into separate static and dynamic signed distance fields and learns motion implicitly through temporal aggregation. Additionally, we introduce a strong self-supervised flow cue derived from features' cosine similarities. We demonstrate the efficacy of our 3D occupancy flow method on SemanticKITTI, KITTI-MOT, and nuScenes.

Problem

• 3D Occupancy Flow
• Self-supervised Learning
• Autonomous Driving
• Motion Estimation
• Scene Understanding

Innovation

• self-supervised learning
• 3D occupancy flow
• signed distance field
• temporal aggregation
• cosine similarity