GUIDE-CoT: Goal-driven and User-Informed Dynamic Estimation for Pedestrian Trajectory using Chain-of-Thought

2025-03-10

AI Summary

This paper addresses two key challenges in pedestrian trajectory prediction: insufficient utilization of visual information and difficulties in end-to-end trajectory generation. To this end, we propose a target-driven, user-controllable dynamic estimation framework. Our method integrates a novel target-oriented visual prompting module to enhance vision–language alignment; a synergistic architecture coupling a vision encoder with a chain-of-thought (CoT) large language model to enable controllable reasoning and interactive editing; and a target-conditioned trajectory decoding mechanism for semantically guided, precise trajectory generation. Evaluated on the ETH/UCY benchmark, our approach achieves state-of-the-art performance, significantly improving prediction plausibility, social interaction awareness, and user controllability. The source code is publicly available.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

While Large Language Models (LLMs) have recently shown impressive results in reasoning tasks, their application to pedestrian trajectory prediction remains challenging due to two key limitations: insufficient use of visual information and the difficulty of predicting entire trajectories. To address these challenges, we propose Goal-driven and User-Informed Dynamic Estimation for pedestrian trajectory using Chain-of-Thought (GUIDE-CoT). Our approach integrates two innovative modules: (1) a goal-oriented visual prompt, which enhances goal prediction accuracy combining visual prompts with a pretrained visual encoder, and (2) a chain-of-thought (CoT) LLM for trajectory generation, which generates realistic trajectories toward the predicted goal. Moreover, our method introduces controllable trajectory generation, allowing for flexible and user-guided modifications to the predicted paths. Through extensive experiments on the ETH/UCY benchmark datasets, our method achieves state-of-the-art performance, delivering both high accuracy and greater adaptability in pedestrian trajectory prediction. Our code is publicly available at https://github.com/ai-kmu/GUIDE-CoT.

Problem

• Enhances pedestrian trajectory prediction accuracy using visual prompts.
• Generates realistic trajectories with a chain-of-thought LLM approach.
• Introduces user-guided modifications for flexible trajectory generation.

Innovation

• Goal-oriented visual prompt enhances prediction accuracy
• Chain-of-Thought LLM generates realistic pedestrian trajectories
• Controllable trajectory generation allows user-guided modifications