IMKD: Intensity-Aware Multi-Level Knowledge Distillation for Camera-Radar Fusion

2025-12-17

AI Summary

Existing radar-camera fusion methods for knowledge distillation directly transfer modality-specific features, leading to modality homogenization and degradation of inherent modality characteristics. To address this challenge in LiDAR-free 3D object detection, we propose an intensity-aware multi-stage knowledge distillation framework. Our method introduces a novel three-stage distillation paradigm: (1) LiDAR-to-radar feature enhancement, (2) geometrically guided distillation from LiDAR to the fusion layer, and (3) intensity-driven camera-radar feature alignment—jointly preserving modality specificity and enabling cross-modal complementarity. Key technical components include intensity-aware attention mechanisms, structural modeling of radar point clouds, and explicit multi-modal feature alignment. Evaluated on nuScenes, our approach achieves 67.0% NDS and 61.0% mAP, substantially outperforming all existing distillation-based radar-camera fusion methods.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

High-performance Radar-Camera 3D object detection can be achieved by leveraging knowledge distillation without using LiDAR at inference time. However, existing distillation methods typically transfer modality-specific features directly to each sensor, which can distort their unique characteristics and degrade their individual strengths. To address this, we introduce IMKD, a radar-camera fusion framework based on multi-level knowledge distillation that preserves each sensor's intrinsic characteristics while amplifying their complementary strengths. IMKD applies a three-stage, intensity-aware distillation strategy to enrich the fused representation across the architecture: (1) LiDAR-to-Radar intensity-aware feature distillation to enhance radar representations with fine-grained structural cues, (2) LiDAR-to-Fused feature intensity-guided distillation to selectively highlight useful geometry and depth information at the fusion level, fostering complementarity between the modalities rather than forcing them to align, and (3) Camera-Radar intensity-guided fusion mechanism that facilitates effective feature alignment and calibration. Extensive experiments on the nuScenes benchmark show that IMKD reaches 67.0% NDS and 61.0% mAP, outperforming all prior distillation-based radar-camera fusion methods. Our code and models are available at https://github.com/dfki-av/IMKD/.

Problem

• Enhances radar-camera 3D detection via multi-level knowledge distillation
• Preserves sensor uniqueness while amplifying complementary strengths
• Uses intensity-aware distillation to improve fused feature representations

Innovation

• Intensity-aware multi-level knowledge distillation for fusion
• Three-stage distillation preserving sensor characteristics
• Intensity-guided fusion mechanism enhancing complementary strengths