Latent Spaces Beyond Synthesis: From GANs to Diffusion Models

2025-10-20

AI Summary

This work investigates the evolutionary mechanisms underlying internal representations in generative vision models, focusing on the paradigm shift from GANs/VAEs to diffusion models, and critically examines the “latent space unity” hypothesis. Method: We propose a theoretical distinction between “strict synthesis” (where a compact latent space fully governs generation) and “generalized synthesis” (where representational tasks are hierarchically and distributedly executed), supported by architectural analysis, layer-wise representation intervention experiments, and interdisciplinary interpretation grounded in media theory. Contribution/Results: Empirical findings reveal that diffusion models decouple semantic and geometric constraints and distribute them across network layers—thereby invalidating the traditional monolithic latent-space metaphor. The study reconceptualizes generative AI not as holistic direct synthesis, but as the emergent outcome of specialized, coordinated subprocesses operating within a hierarchical architecture. This provides a novel theoretical framework and empirical foundation for understanding generative mechanisms.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

This paper examines the evolving nature of internal representations in generative visual models, focusing on the conceptual and technical shift from GANs and VAEs to diffusion-based architectures. Drawing on Beatrice Fazi's account of synthesis as the amalgamation of distributed representations, we propose a distinction between "synthesis in a strict sense", where a compact latent space wholly determines the generative process, and "synthesis in a broad sense," which characterizes models whose representational labor is distributed across layers. Through close readings of model architectures and a targeted experimental setup that intervenes in layerwise representations, we show how diffusion models fragment the burden of representation and thereby challenge assumptions of unified internal space. By situating these findings within media theoretical frameworks and critically engaging with metaphors such as the latent space and the Platonic Representation Hypothesis, we argue for a reorientation of how generative AI is understood: not as a direct synthesis of content, but as an emergent configuration of specialized processes.

Problem

• Analyzing evolving internal representations in generative visual models
• Distinguishing strict vs broad synthesis in GANs and diffusion models
• Challenging unified latent space assumptions through layer interventions

Innovation

• Distinguishes strict vs broad synthesis in models
• Shows diffusion models fragment representation across layers
• Proposes generative AI as emergent specialized processes