Decoding Neighborhood Environments with Large Language Models

2025-05-13

AI Summary

Conventional neighborhood environment assessment relies on labor-intensive field surveys and GIS analysis, incurring high costs and poor scalability; machine learning approaches are similarly constrained by expensive annotation requirements and limited model accessibility. Method: This study pioneers the systematic evaluation of large language models (LLMs) for zero-shot, fine-tuning-free decoding of physical environmental features (e.g., sidewalks, utility poles). We propose a multi-LLM collaborative framework integrating ChatGPT, Gemini, Claude, and Grok, coupled with YOLOv11 object detection (99.13% mAP) and prompt engineering, employing majority voting for consensus inference. Contribution/Results: Our approach achieves >88% accuracy in environmental feature identification—significantly outperforming single-LLM baselines—while eliminating training overhead, reducing cost, and enabling scalable, fully automated neighborhood environment assessment, thereby establishing a paradigm shift from traditional methodologies.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Neighborhood environments include physical and environmental conditions such as housing quality, roads, and sidewalks, which significantly influence human health and well-being. Traditional methods for assessing these environments, including field surveys and geographic information systems (GIS), are resource-intensive and challenging to evaluate neighborhood environments at scale. Although machine learning offers potential for automated analysis, the laborious process of labeling training data and the lack of accessible models hinder scalability. This study explores the feasibility of large language models (LLMs) such as ChatGPT and Gemini as tools for decoding neighborhood environments (e.g., sidewalk and powerline) at scale. We train a robust YOLOv11-based model, which achieves an average accuracy of 99.13% in detecting six environmental indicators, including streetlight, sidewalk, powerline, apartment, single-lane road, and multilane road. We then evaluate four LLMs, including ChatGPT, Gemini, Claude, and Grok, to assess their feasibility, robustness, and limitations in identifying these indicators, with a focus on the impact of prompting strategies and fine-tuning. We apply majority voting with the top three LLMs to achieve over 88% accuracy, which demonstrates LLMs could be a useful tool to decode the neighborhood environment without any training effort.

Problem

• Assessing neighborhood environments efficiently at scale
• Overcoming limitations of traditional resource-intensive methods
• Evaluating LLMs for automated environmental indicator detection

Innovation

• Uses YOLOv11 for 99% accurate environmental detection
• Evaluates four LLMs including ChatGPT and Gemini
• Applies majority voting for 88% accuracy