An LLM-based Delphi Study to Predict GenAI Evolution

2025-02-28

AI Summary

Predicting the evolutionary trajectories of complex dynamic systems under data-scarce conditions remains challenging. This paper introduces the first LLM-driven Delphi paradigm, which simulates expert collective reasoning via iterative prompt engineering and consensus modeling to enable qualitative forecasting of generative AI trends—without requiring real human experts. The method integrates geopolitical dynamics, economic disparities, regulatory frameworks, and ethical dimensions into structured scenario analysis. Compared to conventional Delphi methods, it substantially mitigates respondent fatigue and enables efficient synthesis of multi-perspective insights. Its key contribution lies in embedding group consensus mechanisms directly into the LLM’s inference process—thereby overcoming limitations of knowledge cutoff dates and sensitivity to initial conditions. This yields a reproducible, scalable methodological framework for foresight prediction in highly uncertain domains.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Predicting the future trajectory of complex and rapidly evolving systems remains a significant challenge, particularly in domains where data is scarce or unreliable. This study introduces a novel approach to qualitative forecasting by leveraging Large Language Models to conduct Delphi studies. The methodology was applied to explore the future evolution of Generative Artificial Intelligence, revealing insights into key factors such as geopolitical tensions, economic disparities, regulatory frameworks, and ethical considerations. The results highlight how LLM-based Delphi studies can facilitate structured scenario analysis, capturing diverse perspectives while mitigating issues such as respondent fatigue. However, limitations emerge in terms of knowledge cutoffs, inherent biases, and sensitivity to initial conditions. While the approach provides an innovative means for structured foresight, this method could be also considered as a novel form of reasoning. further research is needed to refine its ability to manage heterogeneity, improve reliability, and integrate external data sources.

Problem

• Predicting future evolution of Generative AI systems.
• Addressing data scarcity and unreliability in forecasting.
• Mitigating respondent fatigue and biases in Delphi studies.

Innovation

• LLM-based Delphi studies for qualitative forecasting
• Structured scenario analysis with diverse perspectives
• Mitigating respondent fatigue in complex predictions