generative artificial intelligence

AI systems that produce new content (text, images, audio, video, code) using generative architectures such as GANs, VAEs, autoregressive transformers and diffusion models; working with generative AI involves dataset curation, model training and conditioning, sampling methods, safety/mitigation techniques, and evaluation using metrics and human studies.

generativeartificialintelligence

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

This work systematically evaluates the applicability of generative AI to scientific image understanding, focusing on text-to-image and image-to-image generation tasks. We propose the first horizontal evaluation framework tailored to scientific imaging scenarios, benchmarking three dominant generative architectures—VAEs, GANs, and diffusion models—across six quantitative dimensions: fidelity, controllability, physical consistency, noise robustness, fine-grained detail accuracy, and domain adaptation efficiency. To address domain-specific requirements, we introduce novel evaluation metrics for generative quality in scientific imaging. Our analysis reveals fundamental trade-offs among key performance indicators across architectures. Furthermore, we identify concrete technical pathways toward enhancing model interpretability. Collectively, these findings provide both theoretical foundations and practical guidelines for the reliable deployment of generative AI in computational imaging, microscopy analysis, and other scientific domains.

Compare architectures: Variational Autoencoders, GANs, Diffusion Models.Discuss challenges and future research in scientific image understanding.Survey state-of-the-art text-to-image and image-to-image generation.

Generative AI for Film Creation: A Survey of Recent Advances

Apr 11, 2025
RZ
Ruihan Zhang
🏛️ Google | University of California, Santa Barbara | MYStudio | Harvard University | Hong Kong University of Science and Technology | Reality Hack | SUNY Buffalo | Onceness | University of Southampton | New York University | Communication University of China | University of Southern California | Dodge College of Film and Media Arts | Pratt Institute | Rubyspot | MIT

This study addresses three core challenges hindering generative AI adoption in film production—character inconsistency, stylistic discontinuity, and motion discontinuity—by proposing the first GenAI technology adoption framework tailored for cinematic workflows. Methodologically, it integrates text-to-image/video diffusion models, Neural Radiance Fields (NeRF), AI-driven avatar generation, 3D synthesis, and multimodal editing techniques to systematically investigate character generation, stylized expression, narrative construction, and live-action–AI content integration. Through empirical artist interviews, key improvement priorities—including controllability, fine-grained editing, and motion optimization—are identified. The primary contributions are: (1) a reusable AI-augmented filmmaking paradigm; (2) a bidirectional roadmap co-evolving technical advancement and artistic practice; and (3) a conceptual shift from AI-as-tool to AI-as-collaborative agent in creative filmmaking.

Addressing artists' challenges in controllability and fine-grained editingExamining GenAI adoption in filmmaking workflows and contributionsExploring strategies for character consistency and stylistic coherence

Must-Read Papers

Most classic and influential ideas
View more

If generative AI is the answer, what is the question?

Sep 07, 2025
AT
Ambuj Tewari
🏛️ University of Michigan

This paper addresses a foundational question in generative AI: What is its intrinsic nature as a distinct machine learning task, and how can generative tasks be formally characterized and theoretically related to prediction, compression, and decision-making? To this end, the authors propose a task-centric research paradigm and develop a unified theoretical framework integrating probabilistic modeling and two-player game theory, rigorously distinguishing density estimation from sampling-based generation. The framework systematically unifies five major generative paradigms—autoregressive models, variational autoencoders, normalizing flows, generative adversarial networks, and diffusion models—and incorporates post-training alignment strategies while embedding socio-ethical considerations. The resulting formal foundation advances the theoretical understanding of generative AI and provides systematic support for responsible AI practices, including privacy-preserving generation, content provenance, and copyright-compliant deployment.

Addressing socially responsible generation including privacy and copyrightExploring generative AI as a distinct machine learning taskSurveying five major generative model families and frameworks

Generative AI, while accelerating digital transformation in government and enterprise sectors, introduces novel security risks—including data leakage, prompt injection, and model misuse—necessitating systematic governance. Method: This study proposes the first comprehensive, lifecycle-oriented risk analysis framework specifically designed for generative AI. It delineates clear responsibility boundaries among users, developers, and operators, enabling proactive security control during development and deployment phases. The framework integrates threat modeling, compliance assessment (aligned with ISO/IEC 27001 and related standards), and scenario-specific risk identification to deliver an actionable risk evaluation methodology. Contribution/Results: The framework empowers organizations to formulate differentiated governance strategies, significantly enhancing the security, controllability, and regulatory compliance of generative AI applications.

Cybersecurity RisksGenerative AI ModelsRisk Management

Generative AI for Vision: A Comprehensive Study of Frameworks and Applications

Jan 29, 2025
FB
Fouad Bousetouane
🏛️ The University of Chicago | 2ndsight.ai

This paper addresses key deployment bottlenecks hindering practical adoption of generative AI for image synthesis—namely, high computational overhead, data bias, and poor alignment with user intent. To tackle these challenges, we propose a structured, input-modality–centric taxonomy that unifies modeling across GANs, diffusion models, and conditional generation paradigms. We systematically categorize core tasks—including image-to-image translation, text-to-image generation, domain adaptation, and multimodal alignment—and conduct an in-depth analysis of architectural design principles and applicability boundaries of representative models such as DALL·E, ControlNet, and DeepSeek Janus-Pro. Furthermore, we establish an industrial-deployment–oriented evaluation framework, explicitly delineating optimization pathways for computational efficiency, bias mitigation, and intent alignment. The resulting methodology provides researchers and practitioners with a theoretically grounded yet practically actionable guide for developing and deploying robust, equitable, and controllable generative image systems.

Artificial IntelligenceComputational CostImage Generation

On the Challenges and Opportunities in Generative AI

Feb 28, 2024
LM
Laura Manduchi
🏛️ ETH Zürich | UC Irvine | University of Tübingen | RPTU Kaiserslautern-Landau | Ruhr-University Bochum | TU Wien | Imperial College London | Johns Hopkins University | LMU Munich | New York University | University of Wisconsin-Madison | Meta AI | UCLA | University of Michigan | Mirelo AI | University of British Columbia | TU Munich

This paper addresses fundamental deficiencies in large language model–based generative AI—specifically, poor reliability, limited generalizability, and weak cross-domain applicability—tracing them to critical bottlenecks including excessive data dependence, insufficient controllability, and the absence of standardized evaluation criteria. Methodologically, it introduces the first four-dimensional analytical framework—encompassing alignment, robustness, interpretability, and accessibility—integrating systematic literature review, paradigm critique, and cross-modal behavioral diagnostics. The framework informs a pragmatic research prioritization roadmap. As a key contribution, the study distills twelve high-priority open problems spanning foundational theory, safety governance, and inclusive deployment. These insights provide both systematic scholarly guidance and actionable reference for advancing generative AI’s theoretical foundations, regulatory frameworks, and equitable real-world implementation.

Highlight unresolved challenges in modern generative AI paradigms.Identify fundamental shortcomings in large-scale generative AI models.Provide insights for developing robust and accessible generative AI solutions.

AI-Generated Content (AIGC) for Various Data Modalities: A Survey

Aug 27, 2023
LG
Lin Geng Foo
🏛️ Singapore University of Technology and Design | Lancaster University

This survey addresses the growing complexity and fragmentation in AI-generated content (AIGC) research across diverse modalities. We systematically review generative methods and cross-modal translation techniques—including text-to-image, audio-to-video, and others—spanning seven modalities: text, image, video, 3D shape/scene/portrait/motion, and audio. Methodologically, we establish the first unified analytical framework covering all modalities, grounded in foundational architectures: GANs, VAEs, diffusion models, autoregressive Transformers, and multimodal alignment paradigms (e.g., CLIP, Flux). Our key contributions include: (i) a novel taxonomy of cross-modal generation paradigms; (ii) a horizontal multimodal comparison framework; (iii) synthesis of 120+ representative works; and (iv) a consolidated analysis of datasets, evaluation metrics, shared challenges, performance bottlenecks, and a comparative performance table. The survey provides systematic guidance for AIGC technology selection, benchmark development, and future research directions.

Artificial Intelligence Generated ContentCross-modal ConversionMultimodal Creation

Latest Papers

What's happening recently
View more

This study addresses the current lack of systematic integration of image-generating generative AI in modeling and simulation. It presents the first comprehensive exploration of text-to-image generation techniques within this domain, proposing tool-agnostic, transferable principles and establishing a localized, reproducible generation pipeline that combines prompt engineering with simulation output mapping. The proposed approach supports diverse applications—including conceptual model representation, visualization of simulation results, generation of instructional materials, and construction of multi-scale model interfaces—thereby offering practitioners a structured knowledge framework to evaluate and adapt this emerging technology. By doing so, it significantly enhances the visual expressiveness and interactive capabilities of simulation systems.

generative AIimage generationmodeling and simulation

While generative AI can enhance learners’ task performance, it does not necessarily foster deep cognitive or metacognitive processing, leading to a disconnect between performance gains and genuine learning outcomes. This study proposes an integrative analytical framework grounded in educational psychology theory, employing cognitive and metacognitive assessment methods to systematically differentiate surface-level performance from deeper learning achievements in AI-supported contexts. The findings reveal the nuanced and complex impact of generative AI on learning quality, underscoring the necessity for educational AI design to move beyond mere task performance and instead prioritize authentic learning mechanisms. This work provides both theoretical grounding and practical guidance for the development of future intelligent educational tools that effectively support meaningful learning.

cognitive processingeducationgenerative AI

Users often treat generative AI as a black box, leading to cognitive biases and misuse. This work proposes an integrative conceptual framework that deconstructs generative AI into interacting components—data, model architecture, product functionality, and user input—and situates them within the historical evolution of computational paradigms. By synthesizing insights from statistical learning theory, anthropomorphic behavioral characteristics, large language model architectures, human–AI interaction modeling, and educational research methodologies, the framework underscores the distinctive role of educational scholars in uncovering latent mechanisms, addressing epistemic uncertainty, and articulating human–AI collaboration dynamics. It offers a coherent conceptual map to support more rigorous experimental design, critical interpretation of AI behaviors, and responsible deployment in educational contexts.

computing paradigmsGenerative AIhuman-like behavior

Latent Spaces Beyond Synthesis: From GANs to Diffusion Models

Oct 20, 2025
LS
Ludovica Schaerf
🏛️ University of Zurich | Max Planck Society

This work investigates the evolutionary mechanisms underlying internal representations in generative vision models, focusing on the paradigm shift from GANs/VAEs to diffusion models, and critically examines the “latent space unity” hypothesis. Method: We propose a theoretical distinction between “strict synthesis” (where a compact latent space fully governs generation) and “generalized synthesis” (where representational tasks are hierarchically and distributedly executed), supported by architectural analysis, layer-wise representation intervention experiments, and interdisciplinary interpretation grounded in media theory. Contribution/Results: Empirical findings reveal that diffusion models decouple semantic and geometric constraints and distribute them across network layers—thereby invalidating the traditional monolithic latent-space metaphor. The study reconceptualizes generative AI not as holistic direct synthesis, but as the emergent outcome of specialized, coordinated subprocesses operating within a hierarchical architecture. This provides a novel theoretical framework and empirical foundation for understanding generative mechanisms.

Analyzing evolving internal representations in generative visual modelsChallenging unified latent space assumptions through layer interventionsDistinguishing strict vs broad synthesis in GANs and diffusion models

This work addresses the lack of intuitive and systematic mathematical introductions in generative AI, which hinders researchers’ understanding of the intrinsic connections and derivation logic among diverse models. By constructing a coherent theoretical pathway, it unifies mainstream approaches—including PCA, variational autoencoders, diffusion models, normalizing flows, autoregressive models, GANs, and their variants—within a common probabilistic and optimization framework. The exposition integrates core tools such as variational inference, optimal transport, energy-based models, and diffusion processes. Presented with both rigor and accessibility, this study elucidates the mathematical foundations of generative modeling, filling a critical gap in pedagogical resources for mathematically novice readers and substantially enhancing the accessibility of foundational principles in generative AI.

generative AIgenerative modelsintuitive understanding

Hot Scholars

PW

Pengfei Wan

Head of Kling Video Generation Models, Kuaishou Technology
Generative ModelsComputer VisionMultimodal AIComputer Graphics
YD

Yilun Du

Harvard University
Artificial IntelligenceMachine LearningRoboticsComputer Vision
ZL

Ziwei Liu

Associate Professor, Nanyang Technological University
Computer VisionMachine LearningComputer Graphics
SL

Sergey Levine

UC Berkeley, Physical Intelligence
Machine LearningRoboticsReinforcement Learning