Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detection

2026-05-10

AI Summary

This work addresses the challenges of modality heterogeneity, spatial misalignment, and inefficient fusion in cross-modal knowledge distillation by proposing a dual-branch architecture. It enhances point cloud representations through 2D semantic-guided voxel optimization (SGVO) and introduces hyperbolic feature transfer (HFT) with geometric constraints in hyperbolic space, complemented by a feature aggregation with geometric optimization (FAGO) mechanism. This integrated approach effectively mitigates semantic loss and spatial distortion during image–point cloud fusion. Notably, this study is the first to incorporate hyperbolic geometry into cross-modal distillation, achieving significant improvements in 3D object detection accuracy across multiple benchmarks—including SUN RGB-D, ARKitScenes, KITTI, and nuScenes—while maintaining computational efficiency.

Abstract

Cross-modal knowledge distillation has emerged as an effective strategy for integrating point cloud and image features in 3D perception tasks. However, the modality heterogeneity, spatial misalignment, and the representation crisis of multiple modalities often limit the efficient of these cross-modal distillation methods. To address these limitations in existing approaches, we propose a hyperbolic constrained cross-modal distillation method for multimodal 3D object detection (HGC-Det). The proposed HGC-Det framework includes an image branch and a point cloud branch to extract semantic features from two different modalities. The point cloud branch comprises three core components: a 2D semantic-guided voxel optimization component (SGVO), a hyperbolic geometry constrained cross-modal feature transfer component (HFT), and a feature aggregation-based geometry optimization component (FAGO). Specifically, the SGVO component adaptively refines the spatial representation of the 3D branch by leveraging semantic cues from the image branch, thereby mitigating the issue of inadequate representation fusion. The HFT component exploits the intrinsic geometric properties of hyperbolic space to alleviate semantic loss during the fusion of high-dimensional image features and low-dimensional point cloud features. Finally, the FAGO compensates for potential spatial feature degradation introduced by the 2D semantic-guided voxel optimization component. Extensive experiments on indoor datasets (SUN RGB-D, ARKitScenes) and outdoor datasets (KITTI, nuScenes) demonstrate that our method achieves a better trade-off between detection accuracy and computational cost.

Problem

• cross-modal distillation
• modality heterogeneity
• spatial misalignment
• representation crisis
• 3D object detection

Innovation

• Hyperbolic Geometry
• Cross-Modal Distillation
• 3D Object Detection
• Semantic-Guided Voxel Optimization
• Feature Transfer