Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning

2026-04-02

AI Summary

This work addresses the fragility and latency inherent in direct trajectory generation by large models by proposing a hierarchical Agentic Fast-Slow Planning framework that decouples perception, reasoning, planning, and control according to their natural timescales. The approach introduces dual bridging mechanisms—“perception-to-decision” and “decision-to-trajectory”—combined with semantic-guided A* search and a memory-augmented feedback module for adaptive parameter tuning, enabling an efficient and interpretable transformation from scene understanding to executable trajectories. Real-time tracking is subsequently handled by model predictive control (MPC). Evaluated in the CARLA simulator, the method reduces lateral deviation by up to 45% and shortens task completion time by over 12% compared to both pure MPC and A*-guided MPC baselines.

Abstract

Large foundation models enable powerful reasoning for autonomous systems, but mapping semantic intent to reliable real-time control remains challenging. Existing approaches either (i) let Large Language Models (LLMs) generate trajectories directly - brittle, hard to verify, and latency-prone - or (ii) adjust Model Predictive Control (MPC) objectives online - mixing slow deliberation with fast control and blurring interfaces. We propose Agentic Fast-Slow Planning, a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales. The framework contains two bridges. Perception2Decision compresses scenes into ego-centric topologies using an on-vehicle Vision-Language Model (VLM) detector, then maps them to symbolic driving directives in the cloud with an LLM decision maker - reducing bandwidth and delay while preserving interpretability. Decision2Trajectory converts directives into executable paths: Semantic-Guided A* embeds language-derived soft costs into classical search to bias solutions toward feasible trajectories, while an Agentic Refinement Module adapts planner hyperparameters using feedback and memory. Finally, MPC tracks the trajectories in real time, with optional cloud-guided references for difficult cases. Experiments in CARLA show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline. Code is available at https://github.com/cjychenjiayi/icra2026_AFSP.

Problem

• Large foundation models
• real-time control
• autonomous systems
• trajectory planning
• semantic reasoning

Innovation

• Agentic Fast-Slow Planning
• Vision-Language Model
• Semantic-Guided A*
• Model Predictive Control
• Hierarchical Autonomous Planning