autonomous driving

This work addresses the challenge of autonomous operation for heterogeneous robotic platforms—specifically aerial and legged robots—in complex environments where GNSS is denied and perceptual conditions are degraded. The authors propose a unified autonomous system architecture that integrates multimodal sensing (LiDAR, radar, vision, and inertial measurements), factor-graph-based SLAM, semantic understanding, scale-adaptive motion planning, and a multi-layer safety mechanism grounded in control barrier functions. For the first time, this framework enables end-to-end coordination among perception, planning, and safety within a single pipeline. Experimental validation on rotorcraft drones and legged robots demonstrates robust performance in navigating, exploring, detecting targets, and conducting inspections under challenging conditions such as smoke and self-similar structures. The associated code and dataset have been publicly released.

To address challenges in high-speed, high-dynamics racing environments—namely, low perception robustness, difficulty in trajectory optimization, and poor simulation-to-reality transfer—this work designs and implements a ROS 2-based autonomous driving software stack for the Formula Student UK Driverless 2025 competition. The method integrates multi-modal sensing (360° LiDAR, ZED2i stereo vision, and GNSS/IMU), incorporates vehicle dynamics modeling, and enforces environmental constraints during trajectory generation. Developed in the CARLA simulator and deployed in real time on a Jetson AGX Orin edge platform, the system achieves stable cone detection within 35 meters and robust closed-loop control. Key contributions include a lightweight perception-planning co-design architecture tailored to racing scenarios, enabling seamless transfer from simulation to physical hardware—including chassis and actuators—and validated via real-vehicle closed-loop testing. This significantly enhances autonomous performance on complex tracks and improves engineering deployability.

High-level autonomous driving (HAD) systems suffer from poor cross-hardware generalization due to multi-modal perception models’ strong dependence on specific sensor hardware configurations. Method: This paper proposes the first sensor data abstraction framework tailored for HAD, systematically defining and implementing unified abstraction interfaces for cameras, LiDAR, and millimeter-wave radar. Integrating signal processing, geometric modeling, and representation learning, the framework enables hardware-agnostic representations for both uni-modal and multi-modal fusion. Contribution/Results: Evaluated on diverse real-world datasets, this work identifies— for the first time—the core challenges and technical pathways for abstracting all three sensor modalities. It establishes a theoretical foundation and architectural blueprint for building scalable, generalizable perception models that transcend hardware-specific constraints, thereby advancing robust, deployment-ready HAD systems.

To address traffic regulation challenges during the transitional period of mixed human-driven and autonomous vehicle operation, this paper proposes a passive driver identification and behavioral profiling method that relies solely on onboard monocular video and vehicle motion-state data—without requiring active vehicle-side identification. We introduce an end-to-end vision–state fusion framework and pioneer a collaborative crowdsourced paradigm for driving behavior discrimination. We release NexusStreet, the first controllable simulation benchmark for this task. Our approach employs a lightweight CNN–LSTM architecture for multimodal temporal modeling, integrated with driving-behavior feature extraction and contrastive learning to enhance robustness under suboptimal sensing conditions. Experiments demonstrate an 80% identification accuracy using video alone, improving to 93% when fused with motion-state data; critically, the model retains strong discriminative capability even under sensor degradation.

This paper addresses the challenges of poor alignment between multimodal large language models (MLLMs) and autonomous driving behavior planning states, as well as weak decision interpretability. To this end, we propose a plug-and-play MLLM-driven closed-loop planning framework. Methodologically: (1) we construct a data engine that aligns driving states with natural-language explanations; (2) we design a standardized motion planning state interface and a multi-sensor (camera/LiDAR/command) fusion encoding mechanism; (3) we integrate the MLLM as a replaceable behavior planning module into Autopilot and Apollo systems, augmenting it with domain-specific driving rules to enhance safety. Evaluated on the CARLA Town05 Long benchmark, replacing the original decision modules yields performance improvements of +3.2 and +4.7 points, respectively. Results demonstrate the framework’s effectiveness, interpretability, and seamless integration capability without system-level modifications.

This work addresses the validation gap between simulation and real-world deployment of autonomous driving algorithms, particularly the lack of efficient, high-fidelity testing platforms for safety-critical scenarios. To bridge this gap, the authors propose a mixed-reality hardware-in-the-loop testing framework that seamlessly integrates physical mobile robots with high-fidelity virtual environments, enabling multimodal sensing, vehicle-to-everything (V2X) communication, and large-scale multi-agent collaboration. A key innovation is the coexistence of physical and virtual agents within a unified architecture, coupled with an online learning controller based on control barrier functions (CBFs) that establishes an integrated perception-planning-control safety assurance mechanism. Experimental results demonstrate that the platform significantly enhances the reliability and efficiency of sim-to-real transfer and validates its effectiveness across diverse safety-critical scenarios.

This work proposes an end-to-end, open-source ROS 2 framework to address key challenges in autonomous drone development, including the sim-to-real deployment gap, complex integration of heterogeneous software and hardware, and prolonged iteration cycles. The framework integrates GPU-accelerated perception modules, a high-fidelity simulation environment, edge computing capabilities, and network communication modeling, while providing unified support for both PX4 and ArduPilot—the two dominant autopilot systems. It achieves, for the first time, full-stack simulation running over 20× faster than real time, significantly accelerating the build–test–deploy cycle for perception-driven drone systems and thereby enhancing both development efficiency and deployment reliability.

Existing autonomous driving datasets lack sufficient diversity, coordination, and cross-domain support, limiting their utility for training multi-agent, multi-sensor systems. To address this gap, this work proposes a modular data generation pipeline built upon the AVstack framework and the CARLA simulator, capable of efficiently producing terabyte-scale, ground-truth-annotated multimodal data. The pipeline encompasses perspectives from ground vehicles, aerial platforms, and infrastructure sensors, and supports flexible single- or multi-agent configurations under controllable, complex scenarios. This approach represents the first scalable, cross-domain collaborative data generation methodology for autonomous driving, substantially enhancing the customization, training efficacy, and practical applicability of perception and sensor fusion models in cooperative autonomous systems.

This work proposes a modular, full-stack autonomous navigation system tailored for lunar surface operations, where GNSS is unavailable and visual conditions are severely degraded. The system integrates lightweight semantic segmentation with stereo visual odometry and employs a factor-graph-based SLAM backend incorporating loop closure detection to achieve globally consistent, high-precision localization. A hierarchical planning architecture is designed, in which high-level path planning encourages loop closures and systematic area coverage, while local navigation leverages arc-based sampling combined with geometric obstacle detection for real-time collision avoidance. Evaluated in high-fidelity lunar simulations, the system demonstrates centimeter-level localization accuracy, high-quality mapping, and strong repeatability, ultimately securing first place in the Lunar Autonomy Challenge finals.

This work addresses the systemic challenges—such as network latency, computational heterogeneity, and multi-tenant contention—that hinder conventional monolithic autonomous driving architectures in V2X cooperative perception and control. The authors propose a modular distributed platform that decouples the autonomous driving stack into components flexibly deployable across vehicles, roadside units, and edge/cloud infrastructure. For the first time, the platform enables synchronized instrumentation at the model, system, and task levels. It supports trajectory-driven network and workload simulation, dataset-driven benchmarking of distributed inference, and end-to-end performance evaluation. Experiments demonstrate that vehicle-to-vehicle intent messages offer superior safety compared to cloud-based perception, and roadside assistance ensures safety when not overloaded. The platform is open-sourced to facilitate reproducible research in cooperative autonomous driving.