On the Challenges and Opportunities in Generative AI

2024-02-28

arXiv.org

AI Summary

This paper addresses fundamental deficiencies in large language model–based generative AI—specifically, poor reliability, limited generalizability, and weak cross-domain applicability—tracing them to critical bottlenecks including excessive data dependence, insufficient controllability, and the absence of standardized evaluation criteria. Methodologically, it introduces the first four-dimensional analytical framework—encompassing alignment, robustness, interpretability, and accessibility—integrating systematic literature review, paradigm critique, and cross-modal behavioral diagnostics. The framework informs a pragmatic research prioritization roadmap. As a key contribution, the study distills twelve high-priority open problems spanning foundational theory, safety governance, and inclusive deployment. These insights provide both systematic scholarly guidance and actionable reference for advancing generative AI’s theoretical foundations, regulatory frameworks, and equitable real-world implementation.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervised learning paradigms, recent large-scale generative models show tremendous promise in synthesizing high-resolution images and text, as well as structured data such as videos and molecules. However, we argue that current large-scale generative AI models exhibit several fundamental shortcomings that hinder their widespread adoption across domains. In this work, our objective is to identify these issues and highlight key unresolved challenges in modern generative AI paradigms that should be addressed to further enhance their capabilities, versatility, and reliability. By identifying these challenges, we aim to provide researchers with insights for exploring fruitful research directions, thus fostering the development of more robust and accessible generative AI solutions.

Problem

• Identify fundamental shortcomings in large-scale generative AI models.
• Highlight unresolved challenges in modern generative AI paradigms.
• Provide insights for developing robust and accessible generative AI solutions.

Innovation

• Scalable unsupervised learning paradigms
• High-resolution image and text synthesis
• Structured data generation like videos