CODEI: Resource-Efficient Task-Driven Co-Design of Perception and Decision Making for Mobile Robots Applied to Autonomous Vehicles

2025-03-13

AI Summary

Urban autonomous vehicles (AVs) face significant challenges in hardware-software co-design due to tight resource constraints and stringent safety requirements. Method: This paper proposes a task-driven perception-decision joint optimization framework. It introduces (1) an occupancy query mechanism—first of its kind—to quantitatively characterize motion planning’s perceptual requirements, thereby bridging task objectives and perception performance; (2) the first end-to-end, embodied-intelligence-oriented integer linear programming (ILP) model that jointly optimizes robot morphology, sensors, compute units, perception algorithms, and planners; and (3) a multi-dimensional perception evaluation based on false negative/positive rates to enable rigorous resource-safety trade-off analysis. Results: Evaluated on urban AV scenarios, the framework reveals that resource constraints dominantly govern sensor selection: cameras favor cost-efficiency and lightweight deployment, whereas LiDAR improves energy and computational efficiency. Overall, it reduces redundant resource consumption while rigorously maintaining task-level safety boundaries.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper discusses the integration challenges and strategies for designing mobile robots, by focusing on the task-driven, optimal selection of hardware and software to balance safety, efficiency, and minimal usage of resources such as costs, energy, computational requirements, and weight. We emphasize the interplay between perception and motion planning in decision-making by introducing the concept of occupancy queries to quantify the perception requirements for sampling-based motion planners. Sensor and algorithm performance are evaluated using False Negative Rates (FPR) and False Positive Rates (FPR) across various factors such as geometric relationships, object properties, sensor resolution, and environmental conditions. By integrating perception requirements with perception performance, an Integer Linear Programming (ILP) approach is proposed for efficient sensor and algorithm selection and placement. This forms the basis for a co-design optimization that includes the robot body, motion planner, perception pipeline, and computing unit. We refer to this framework for solving the co-design problem of mobile robots as CODEI, short for Co-design of Embodied Intelligence. A case study on developing an Autonomous Vehicle (AV) for urban scenarios provides actionable information for designers, and shows that complex tasks escalate resource demands, with task performance affecting choices of the autonomy stack. The study demonstrates that resource prioritization influences sensor choice: cameras are preferred for cost-effective and lightweight designs, while lidar sensors are chosen for better energy and computational efficiency.

Problem

• Optimize hardware and software for mobile robots
• Balance safety, efficiency, and resource usage
• Integrate perception and motion planning for decision-making

Innovation

• Task-driven hardware and software co-design
• Occupancy queries for perception and motion planning
• Integer Linear Programming for sensor and algorithm optimization