PIE: Perception and Interaction Enhanced End-to-End Motion Planning for Autonomous Driving

2025-09-22

AI Summary

End-to-end motion planning suffers from insufficient scene understanding and interaction prediction, limiting its reliability for autonomous driving. To address this, we propose a novel end-to-end planning framework that unifies perception, reasoning, and interactive modeling. Our approach features a bidirectional Mamba-based multimodal fusion architecture to mitigate feature compression loss; an inference-enhanced decoder coupled with an action-motion interaction module for compliant anchor selection and adaptive trajectory optimization; and an integrated pipeline incorporating camera-LiDAR inputs, a Mamba backbone, a Mixture-of-Experts (MoE) architecture, and a state-prediction-driven interaction mechanism. Evaluated on the NAVSIM benchmark, our method achieves 88.9 PDM and 85.6 EPDM—surpassing prior state-of-the-art—without model ensembling or data augmentation. It significantly improves trajectory quality and scene compliance.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

End-to-end motion planning is promising for simplifying complex autonomous driving pipelines. However, challenges such as scene understanding and effective prediction for decision-making continue to present substantial obstacles to its large-scale deployment. In this paper, we present PIE, a pioneering framework that integrates advanced perception, reasoning, and intention modeling to dynamically capture interactions between the ego vehicle and surrounding agents. It incorporates a bidirectional Mamba fusion that addresses data compression losses in multimodal fusion of camera and LiDAR inputs, alongside a novel reasoning-enhanced decoder integrating Mamba and Mixture-of-Experts to facilitate scene-compliant anchor selection and optimize adaptive trajectory inference. PIE adopts an action-motion interaction module to effectively utilize state predictions of surrounding agents to refine ego planning. The proposed framework is thoroughly validated on the NAVSIM benchmark. PIE, without using any ensemble and data augmentation techniques, achieves an 88.9 PDM score and 85.6 EPDM score, surpassing the performance of prior state-of-the-art methods. Comprehensive quantitative and qualitative analyses demonstrate that PIE is capable of reliably generating feasible and high-quality ego trajectories.

Problem

• Enhancing scene understanding and prediction for autonomous driving motion planning
• Addressing data compression losses in multimodal camera-LiDAR fusion
• Improving ego vehicle trajectory planning using surrounding agent predictions

Innovation

• Bidirectional Mamba fusion for multimodal sensor integration
• Mamba-Mixture-of-Experts decoder for adaptive trajectory inference
• Action-motion interaction module refining ego planning with predictions