Leveraging ChatGPT's Multimodal Vision Capabilities to Rank Satellite Images by Poverty Level: Advancing Tools for Social Science Research

2025-01-24

AI Summary

Conventional poverty assessment relies heavily on costly, labor-intensive field surveys, limiting scalability and timeliness—especially at fine-grained administrative levels such as villages. Method: This study introduces a zero-shot pairwise satellite image comparison paradigm leveraging commercial multimodal large language models (e.g., GPT-4V) to directly infer village-level poverty rankings without model fine-tuning or labeled training data. It integrates geospatial semantic reasoning to interpret visual cues indicative of socioeconomic conditions. Contribution/Results: The approach achieves high agreement with domain experts in village-level poverty ranking (Cohen’s κ = 0.82), demonstrating—for the first time—the viability of off-the-shelf vision-language models for socioeconomic remote sensing inference. By bridging large language models and socioeconomic fieldwork, this work establishes a novel, low-cost, scalable, and interpretable paradigm for global poverty dynamics monitoring.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper investigates the novel application of Large Language Models (LLMs) with vision capabilities to analyze satellite imagery for village-level poverty prediction. Although LLMs were originally designed for natural language understanding, their adaptability to multimodal tasks, including geospatial analysis, has opened new frontiers in data-driven research. By leveraging advancements in vision-enabled LLMs, we assess their ability to provide interpretable, scalable, and reliable insights into human poverty from satellite images. Using a pairwise comparison approach, we demonstrate that ChatGPT can rank satellite images based on poverty levels with accuracy comparable to domain experts. These findings highlight both the promise and the limitations of LLMs in socioeconomic research, providing a foundation for their integration into poverty assessment workflows. This study contributes to the ongoing exploration of unconventional data sources for welfare analysis and opens pathways for cost-effective, large-scale poverty monitoring.

Problem

• Satellite Imagery Analysis
• Economic Condition Assessment
• Large Language Models

Innovation

• ChatGPT
• Image Recognition
• Poverty Mapping