Remote-Sensing City Layout Extraction with MLLM

2026-08-17

AI Summary

This study addresses the lack of editable, structured representations in remote sensing-based urban layout extraction by proposing Code-as-City, a paradigm that formulates layout extraction as constrained code generation. Leveraging multimodal large language models, the approach progressively reconstructs roads and buildings while integrating semantic priors with deterministic normalization to enable synchronized generation and editing of 2D semantic projections and 3D layouts. Evaluated on the CityLayout-100 dataset, the method achieves 41.1% mIoU and 48.3% global IoU, demonstrating its effectiveness. These results validate the feasibility and novelty of transforming visual observations into inspectable, editable urban code, thereby bridging the gap between perceptual remote sensing analysis and programmable urban modeling.

Abstract

Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.

Problem

• Remote-Sensing City Layout Extraction
• Executable Urban Layout
• Code-as-City
• 3D City Layout Generation

Innovation

• Code-as-City
• Multimodal Large Language Model
• Constrained Code Generation
• City Layout Extraction
• Synchronized Regeneration