Digital Twin Buildings: 3D Modeling, GIS Integration, and Visual Descriptions Using Gaussian Splatting, ChatGPT/Deepseek, and Google Maps Platforms

2025-02-09

AI Summary

This paper addresses three key challenges in monolithic building-level digital twin construction: low modeling efficiency, weak semantic understanding, and poor GIS integration. To tackle these, we propose an end-to-end fusion framework comprising three components: (1) a Gaussian Splatting–based pipeline enabling second-level 3D reconstruction from street-view imagery; (2) a multi-agent large language model architecture—integrating ChatGPT-4o and DeepSeek-V3/R1—to generate natural language–driven visual semantic descriptions; and (3) seamless GIS integration via Google Maps Platform API and spatial analysis techniques, supporting address/coordinate/ZIP-code–driven automated modeling, descriptive visualization, and dynamic cloud-map synchronization. To our knowledge, this is the first framework to deeply couple geometric reconstruction, semantic interpretation, and geospatial services. Experimental results demonstrate significant improvements over conventional methods in modeling speed, semantic description accuracy, and cross-platform interoperability.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Urban digital twins are virtual replicas of cities that use multi-source data and data analytics to optimize urban planning, infrastructure management, and decision-making. Towards this, we propose a framework focused on the single-building scale. By connecting to cloud mapping platforms such as Google Map Platforms APIs, by leveraging state-of-the-art multi-agent Large Language Models data analysis using ChatGPT(4o) and Deepseek-V3/R1, and by using our Gaussian Splatting-based mesh extraction pipeline, our Digital Twin Buildings framework can retrieve a building's 3D model, visual descriptions, and achieve cloud-based mapping integration with large language model-based data analytics using a building's address, postal code, or geographic coordinates.

Problem

• Develop 3D models for buildings
• Integrate GIS and cloud mapping
• Analyze data with large language models

Innovation

• Gaussian Splatting mesh extraction
• ChatGPT/Deepseek data analysis
• Google Maps API integration