ParkingTransformer: LLM-Enhanced End-to-End Trajectory Planning for Autonomous Parking

2026-06-12

AI Summary

This work addresses the limitations of existing end-to-end autonomous parking methods, which often lack high-level semantic understanding and interpretability, hindering seamless long-range parking. The authors propose ParkingTransformer, a novel framework that integrates multi-view perception with the scene comprehension capabilities of large language models (LLMs). By enabling direct interaction between trajectory queries and implicit state features from the LLM with raw sensor data and historical context, the method achieves end-to-end trajectory planning without relying on dense bird’s-eye-view representations. Key innovations include 3D positional encoding to enhance the LLM’s spatial geometric awareness, a fixed-window streaming mechanism for efficient long-horizon temporal processing, and a coarse-to-fine decoding strategy for progressive trajectory refinement. Evaluated in CARLA simulation and real-world vehicle platforms, the approach achieves a driving score of 61.32 and an average success rate of 88.70%, demonstrating its effectiveness and practicality.

Abstract

End-to-end autonomous parking has emerged as a critical task within the realm of autonomous driving. However, existing methods suffer from black-box characteristics, lacking high-level semantic understanding and interpretability, which impedes the realization of seamless long-distance autonomous parking from the road to the target spot. To address these limitations, we propose ParkingTransformer, a novel framework that leverages multi-view perception and the scene understanding capability of Large Language Models (LLMs). By combining trajectory queries with LLMs implicit state features, our method interacts directly with historical information and raw sensor data to output planning trajectories, eliminating the need for dense Bird's-View (BEV) representations. To compensate for the inadequate spatial reasoning ability of LLMs, we introduce 3D positional encoding to explicitly inject spatial geometric awareness. Furthermore, a fixed-window streaming mechanism is designed for historical information processing, significantly improving long-term temporal processing efficiency and inference speed. Additionally, a coarse-to-fine decoding strategy is employed to progressively enhance trajectory precision. Extensive closed-loop experiments are conducted on the CARLA simulator and real-world vehicle platforms. The results demonstrate that our method achieves a driving score of 61.32 in CARLA simulator and an average success rate of 88.70% in real-world experiments, validating the feasibility and effectiveness of the proposed algorithms.

Problem

• autonomous parking
• end-to-end trajectory planning
• semantic understanding
• interpretability
• long-distance parking

Innovation

• LLM-enhanced planning
• End-to-end parking
• 3D positional encoding
• Streaming historical processing
• Coarse-to-fine decoding