The Unified Autonomy Stack: Toward a Blueprint for Generalizable Robot Autonomy

2026-05-12

AI Summary

This work addresses the challenge of autonomous operation for heterogeneous robotic platforms—specifically aerial and legged robots—in complex environments where GNSS is denied and perceptual conditions are degraded. The authors propose a unified autonomous system architecture that integrates multimodal sensing (LiDAR, radar, vision, and inertial measurements), factor-graph-based SLAM, semantic understanding, scale-adaptive motion planning, and a multi-layer safety mechanism grounded in control barrier functions. For the first time, this framework enables end-to-end coordination among perception, planning, and safety within a single pipeline. Experimental validation on rotorcraft drones and legged robots demonstrates robust performance in navigating, exploring, detecting targets, and conducting inspections under challenging conditions such as smoke and self-similar structures. The associated code and dataset have been publicly released.

Abstract

We introduce and open-source the Unified Autonomy Stack, a system-level solution that enables resilient autonomy across diverse aerial and ground robot morphologies. The architecture centers on three synergistic modules -- multi-modal perception, multi-behavior planning, and multi-layered safe navigation -- that together deliver comprehensive mission autonomy. The stack fuses data from LiDAR, radar, vision, and inertial sensing, enabling (a) robust localization and mapping through factor graph-based fusion, (b) semantic scene understanding, (c) motion and informative path planning through sampling-based techniques adaptive across spatial scales, as well as (d) multi-layered safe navigation both through planning on the online reconstructed map and deep learning-driven exteroceptive policies alongside last-resort safety filters using control barrier functions. The resulting behaviors include safe GNSS-denied navigation into unknown and perceptually-degraded regions, exploration of complex environments, object discovery, and efficient inspection planning. The stack has been field-tested and validated on both aerial (rotorcraft) and ground (legged) robots operating in a host of demanding environments, including self-similar and smoke-filled settings, with complex geometries and high obstacle clutter. These tests demonstrate resilient performance in challenging conditions. To facilitate ease of adoption, we open-source the implementation alongside supporting documentation, validation, and evaluation datasets https://github.com/ntnu-arl/unified_autonomy_stack. A video giving the overview of the paper and the field experiments is available at https://youtu.be/l8Su8OXsM-E.

Problem

• robot autonomy
• generalizable autonomy
• perceptually-degraded environments
• GNSS-denied navigation
• multi-robot systems

Innovation

• Unified Autonomy Stack
• multi-modal perception
• multi-behavior planning
• safe navigation
• factor graph fusion