sensor systems

To address the insufficient robustness of single-sensor perception for autonomous vehicles under adverse weather and complex urban conditions, this paper systematically investigates multimodal sensor fusion, unifying data-level, feature-level, and decision-level fusion paradigms within a coherent formalism. We propose a deep learning–based cross-modal alignment and representation learning framework that, for the first time, integrates vision-language models (VLMs) and large language models (LLMs) into the sensor fusion pipeline—thereby enhancing adaptability and uncertainty modeling in end-to-end autonomous driving. We establish a comprehensive evaluation framework across major benchmarks—including nuScenes, BDD100K, and Oxford Radar RobotCar—and demonstrate significant improvements in object detection and semantic segmentation accuracy under challenging conditions such as rain, fog, and nighttime.

High-level autonomous driving (HAD) systems suffer from poor cross-hardware generalization due to multi-modal perception models’ strong dependence on specific sensor hardware configurations. Method: This paper proposes the first sensor data abstraction framework tailored for HAD, systematically defining and implementing unified abstraction interfaces for cameras, LiDAR, and millimeter-wave radar. Integrating signal processing, geometric modeling, and representation learning, the framework enables hardware-agnostic representations for both uni-modal and multi-modal fusion. Contribution/Results: Evaluated on diverse real-world datasets, this work identifies— for the first time—the core challenges and technical pathways for abstracting all three sensor modalities. It establishes a theoretical foundation and architectural blueprint for building scalable, generalizable perception models that transcend hardware-specific constraints, thereby advancing robust, deployment-ready HAD systems.

To address the limited robustness and sensor-specific modeling dependency of LiDAR-inertial odometry under diverse sensor configurations and operational scenarios, this paper proposes a generic fusion framework that requires no prior sensor modeling. Methodologically, it employs a simplified IMU motion model for inertial integration—eliminating both feature extraction and preintegration—and introduces a direct scan-to-map LiDAR registration with a novel regularization mechanism to improve convergence stability. The key contributions are: (1) a unified configuration enabling cross-platform deployment (e.g., urban driving, natural environments) and cross-sensor compatibility (various LiDAR/IMU models); (2) experimental validation on multiple real-world robotic platforms demonstrating high accuracy, strong robustness, and real-time performance; and (3) open-sourced implementation.

In autonomous driving, trajectory prediction reliability is severely degraded by sensor staleness—temporal misalignment among multi-sensor data caused by transmission delays. To address this, we propose a robust asynchronous multimodal fusion framework. Our method introduces point-level temporal offset features to explicitly model the time lag of LiDAR and radar relative to camera inputs; designs a vehicle-observed stale-pattern-based data augmentation strategy that is model-agnostic and plug-and-play; and integrates perspective-domain detection, cross-modal temporal alignment, temporal-aware feature encoding, and simulation-enhanced training. Evaluated under both synchronized and diverse stale conditions—including severe unimodal latency—the framework demonstrates stable, state-of-the-art performance, significantly outperforming baselines while maintaining safety-critical trajectory prediction accuracy even under extreme sensor delay.

To address challenges in high-risk road segments (e.g., construction zones), including severe viewpoint distortion, occlusion, geometric complexity, and high deployment costs, this paper proposes a lightweight multi-sensor fusion framework integrating roadside cameras and LiDAR—augmented by radar and RTK-GPS—for low-cost, scalable vehicle detection and high-precision localization. A novel late-fusion strategy based on Kalman filtering is introduced to enable sensor complementarity, fault tolerance against individual sensor failures, and enhanced trajectory consistency. In a co-simulation environment, the method reduces longitudinal positioning error by 70% and achieves lateral accuracy of 1–3 meters. Field validation confirms strong alignment between fused trajectories and ground-truth references, demonstrating robustness under real-world conditions and practical feasibility for large-scale deployment.

This work uncovers a novel security vulnerability in autonomous vehicle sensors—particularly LiDAR—under partial-information attacks: adversaries can stealthily inject network-layer Trojans while respecting sensor integrity constraints. We first identify LiDAR’s critical fragility in multi-sensor fusion pipelines. To address this, we propose a probabilistic data asymmetry monitor and design a trajectory-level 3D LiDAR–monocular camera fusion method (T2T-3DLM) to achieve robust, security-aware perception. Evaluated on AV simulation platforms and real-world datasets, our approach integrates adversarial modeling and integrity constraint analysis. Experiments demonstrate that the proposed solution significantly reduces Trojan attack success rates, confirms LiDAR-based attacks pose greater security risks than camera-based ones, and enhances system robustness substantially—without compromising functional performance.

In radar–LiDAR–inertial SLAM, asynchronous sensor operation causes excessive state node proliferation and prohibitive optimization overhead. To address this, we propose an IMU preintegration-based radar factor modeling method: IMU preintegration is employed to propagate LiDAR states to radar measurement timestamps, eliminating the need for dedicated state nodes per radar frame and reducing state variable generation frequency by 50%. This work is the first to incorporate IMU preintegration into radar factor design, enabling tightly coupled asynchronous fusion while preserving the original LiDAR node frequency—thereby significantly compressing the factor graph size. Experiments on an embedded single-board computer demonstrate a 56% reduction in overall optimization time, with absolute pose accuracy matching that of conventional synchronous approaches. The method thus achieves an effective trade-off between real-time performance and localization accuracy.

This work addresses the degradation of tracking performance in asynchronous multi-sensor systems, where consecutive missed detections from high-frequency sensors can disrupt trajectories maintained by low-frequency sensors—a limitation rooted in the conventional assumption of globally uniform observability. To overcome this, the authors introduce a DetectorContext abstraction within the Stone Soup framework, which dynamically models detection probability and clutter intensity as functions of both target state and perceptual context during hypothesis generation. This context-aware sensor modeling enhances trajectory stability in asynchronous, partially overlapping sensor configurations without requiring modifications to the update equations of existing probabilistic trackers. Experimental results on radar–LiDAR asynchronous data demonstrate significant improvements in HOTA and GOSPA metrics, confirming restored fusion performance without introducing spurious tracks.

High-precision online estimation algorithms for robotics are highly sensitive to sensor timestamp accuracy; however, existing synchronization solutions struggle to simultaneously achieve real-time operation, low cost, and high temporal precision. To address this, we propose a real-time, trigger-based time synchronization system built on commodity hardware. Our approach employs a hardware-triggered mechanism to jointly schedule heterogeneous sensors operating at different frequencies, and integrates an enhanced clock synchronization protocol with nanosecond-resolution timestamping to ensure precise coordination between sensors and the onboard computer. Crucially, the system eliminates reliance on expensive dedicated timing hardware, thereby substantially mitigating the impact of timing errors on online estimation. Experimental evaluation on a physical robot platform demonstrates sub-microsecond synchronization accuracy, along with significant improvements in both estimation robustness and real-time performance.

To address the high cost, computational overhead, and poor robustness of conventional solutions for urban roadside perception, this paper proposes a CPU-only real-time traffic participant tracking framework. The framework leverages a single-line LiDAR sensor and an edge computing unit, integrating a lightweight extended Kalman filter, 1D grid-map-based Bayesian state update, footprint lookup table-driven fine-grained classification, and a trajectory-age- and bounding-box-consistency-based existence criterion to jointly estimate position, velocity, class, and existence. Evaluated in urban-like dynamic scenarios, the end-to-end system achieves 99.88% message processing latency ≤100 ms, high detection accuracy, and stable performance under simulated wind-induced vibration. This work presents the first empirical validation of a GPU-free, CPU-only architecture for large-scale roadside perception—demonstrating feasibility for real-time, robust, and low-cost deployment.

This work addresses the critical challenge of jointly designing sensor query rates and noise covariance under resource and cost constraints to meet prescribed trajectory estimation accuracy requirements. It presents the first formalization of this problem as a unified optimization model, leveraging semidefinite programming (SDP) within the Kalman filter error covariance framework to simultaneously optimize measurement scheduling and noise parameters. The proposed approach efficiently determines whether a given accuracy target is achievable and, when feasible, synthesizes a corresponding implementation strategy. Experimental validation demonstrates that the computed sensor configurations consistently attain the desired accuracy in both simulated and real-world scenarios, while also reliably identifying infeasible accuracy demands.