TFusionOcc: Student's t-Distribution Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Prediction

2026-02-06

AI Summary

This work addresses the limitations of existing 3D semantic occupancy prediction methods, which rely on voxel or Gaussian representations and struggle to efficiently model fine-grained geometric and semantic structures in autonomous driving scenes. To overcome this, the authors propose an object-centric, multi-sensor fusion framework that employs deformable superquadrics—augmented with inverse warping—as flexible geometric primitives. The representation is further enhanced by integrating Student’s t-distribution within a T-mixture model (TMM) to improve expressiveness for complex shapes and robustness under sensor degradation. Through a multi-stage fusion strategy, the method achieves state-of-the-art performance on the nuScenes benchmark and significantly outperforms existing approaches across various corruption scenarios in nuScenes-C, involving both camera and LiDAR failures.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: 3D Computer Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

3D semantic occupancy prediction enables autonomous vehicles (AVs) to perceive fine-grained geometric and semantic structure of their surroundings from onboard sensors, which is essential for safe decision-making and navigation. Recent models for 3D semantic occupancy prediction have successfully addressed the challenge of describing real-world objects with varied shapes and classes. However, the intermediate representations used by existing methods for 3D semantic occupancy prediction rely heavily on 3D voxel volumes or a set of 3D Gaussians, hindering the model's ability to efficiently and effectively capture fine-grained geometric details in the 3D driving environment. This paper introduces TFusionOcc, a novel object-centric multi-sensor fusion framework for predicting 3D semantic occupancy. By leveraging multi-stage multi-sensor fusion, Student's t-distribution, and the T-Mixture model (TMM), together with more geometrically flexible primitives, such as the deformable superquadric (superquadric with inverse warp), the proposed method achieved state-of-the-art (SOTA) performance on the nuScenes benchmark. In addition, extensive experiments were conducted on the nuScenes-C dataset to demonstrate the robustness of the proposed method in different camera and lidar corruption scenarios. The code will be available at: https://github.com/DanielMing123/TFusionOcc

Problem

• 3D semantic occupancy prediction
• multi-sensor fusion
• geometric representation
• autonomous driving
• fine-grained geometry

Innovation

• Student's t-distribution
• T-Mixture Model
• deformable superquadric
• object-centric fusion
• 3D semantic occupancy