Graphical-Probabilistic Modeling of Generative Flows in LLM-Native Software Systems

2026-06-14

AI Summary

Current LLM-native software development lacks structured methodologies that support design-level reasoning, relying predominantly on low-level prompt engineering. This work proposes “generative networks”—a novel framework grounded in probabilistic graphical models—that formally adapts graph-based probabilistic modeling to LLM-driven systems. For the first time, it provides a formal characterization of generative flows and their system-level properties, such as stochasticity, prompt dependency, and emergent behaviors. By doing so, the framework establishes a theoretical foundation and offers modeling tools for the documented design, declarative specification of properties, and systematic analysis of LLM-native systems.

Abstract

Engineering LLM-native software remains a challenging and immature field. Current practice is largely exploratory, relying on experimentation and heuristic techniques such as prompting and context engineering. These, however, are low-level and lack the principled structure needed to support design-level reasoning or analysis. In contrast, traditional software engineering leverages modularity and abstraction to communicate and analyze system behavior. To bring similar rigor to LLM-native development, we propose methods for documenting generative flows and for stating properties of LLM-based software designs. Such methods must account for the stochastic, prompt-dependent behavior of large language models while remaining expressive enough to capture emergent phenomena. Our initial approach is based on graphical probabilistic models, tailored to capture phenomena characteristic of LLM-native systems. This framework -- what we term Generation Networks -- aims to provide a foundation for principled reasoning about generative interactions and system-level properties in LLM-centric software architectures.

Problem

• LLM-native software
• generative flows
• software design
• probabilistic modeling
• system-level reasoning

Innovation

• Graphical Probabilistic Models
• Generative Flows
• LLM-Native Software
• Generation Networks
• Prompt-Dependent Behavior