HiLO: High-Level Object Fusion for Autonomous Driving using Transformers

2025-06-03

AI Summary

To address the high computational cost and insufficient cross-scenario robustness of multi-sensor object-level fusion in autonomous driving, this paper proposes HiLO—the first Transformer-based higher-order object fusion framework. HiLO tackles these challenges through three key innovations: (1) the first integration of Transformers into object-level fusion, enabling end-to-end learnable cross-modal feature alignment; (2) coupling with an enhanced adaptive Kalman filter (AKF) to support dynamic confidence-weighted fusion; and (3) a domain-agnostic architecture designed for both urban and highway scenarios. Evaluated on a large-scale, in-house real-world dataset, HiLO achieves a 25.9-percentage-point improvement in F1 score and a 6.1-percentage-point gain in mean IoU over state-of-the-art baselines, demonstrating significantly enhanced generalization capability and computational efficiency.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The fusion of sensor data is essential for a robust perception of the environment in autonomous driving. Learning-based fusion approaches mainly use feature-level fusion to achieve high performance, but their complexity and hardware requirements limit their applicability in near-production vehicles. High-level fusion methods offer robustness with lower computational requirements. Traditional methods, such as the Kalman filter, dominate this area. This paper modifies the Adapted Kalman Filter (AKF) and proposes a novel transformer-based high-level object fusion method called HiLO. Experimental results demonstrate improvements of $25.9$ percentage points in $ extrm{F}_1$ score and $6.1$ percentage points in mean IoU. Evaluation on a new large-scale real-world dataset demonstrates the effectiveness of the proposed approaches. Their generalizability is further validated by cross-domain evaluation between urban and highway scenarios. Code, data, and models are available at https://github.com/rst-tu-dortmund/HiLO .

Problem

• Improving sensor fusion robustness for autonomous driving
• Reducing computational complexity in high-level fusion methods
• Enhancing object fusion accuracy using transformer-based approach

Innovation

• Modified Adapted Kalman Filter (AKF)
• Transformer-based high-level object fusion
• Improved F1 score and mean IoU