🤖 AI Summary
Traditional road detection methods suffer from poor generalization in unseen scenarios and lack semantic interpretability, undermining the reliability of C-ITS traffic advisories. To address this, we propose a multi-agent collaborative framework integrating Gemini multimodal large language models with visual perception for real-time road situation understanding and standardized C-ITS message generation. Specialized agents perform situational recognition, distance estimation, decision reasoning, and structured message generation, enhancing both system interpretability and semantic reasoning capability. Evaluated on a custom road dataset comprising 103 images, our approach achieves 100% situation detection rate and fully correct syntactic message generation. Experiments demonstrate that Gemini-2.0-Flash outperforms Gemini-2.5-Flash in both accuracy and inference latency. This work establishes a novel, interpretable, and scalable multimodal semantic understanding paradigm for intelligent transportation systems.
📝 Abstract
Conventional road-situation detection methods achieve strong performance in predefined scenarios but fail in unseen cases and lack semantic interpretation, which is crucial for reliable traffic recommendations. This work introduces a multi-agent AI framework that combines multimodal large language models (MLLMs) with vision-based perception for road-situation monitoring. The framework processes camera feeds and coordinates dedicated agents for situation detection, distance estimation, decision-making, and Cooperative Intelligent Transport System (C-ITS) message generation. Evaluation is conducted on a custom dataset of 103 images extracted from 20 videos of the TAD dataset. Both Gemini-2.0-Flash and Gemini-2.5-Flash were evaluated. The results show 100% recall in situation detection and perfect message schema correctness; however, both models suffer from false-positive detections and have reduced performance in terms of number of lanes, driving lane status and cause code. Surprisingly, Gemini-2.5-Flash, though more capable in general tasks, underperforms Gemini-2.0-Flash in detection accuracy and semantic understanding and incurs higher latency (Table II). These findings motivate further work on fine-tuning specialized LLMs or MLLMs tailored for intelligent transportation applications.