MMUEChange: A generalized LLM agent framework for intelligent multi-modal urban environment change analysis

2026-01-09

Applied Soft Computing

AI Summary

This work proposes the first modular multimodal large language model (LLM) agent framework tailored for urban change analysis, addressing the limitations of existing approaches that rely on single-modality inputs and rigid pipelines, which struggle to effectively integrate heterogeneous multisource data. The framework introduces a modality controller to enable dynamic intra- and cross-modal alignment, flexibly incorporating remote sensing imagery, nighttime light data, and textual information while mitigating LLM hallucinations. Evaluated on real-world urban case studies, the approach achieves a 46.7% improvement in task success rate over the strongest baseline, significantly enhancing semantic understanding, reasoning capabilities, and policy relevance. It successfully uncovers complex urban dynamics, including green space transformation in New York, water pollution diffusion in Hong Kong, and landfill evolution in Shenzhen.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Problem

• urban environment change
• multi-modal analysis
• remote sensing change detection
• heterogeneous urban data
• sustainable development

Innovation

• multi-modal agent
• urban environment change
• modality alignment
• LLM framework
• heterogeneous data integration