Natural Building Blocks for Structured World Models: Theory, Evidence, and Scaling

2025-11-03

AI Summary

The field of world modeling has long suffered from a lack of unified theoretical foundations, resulting in fragmented architectures and poor interpretability. Method: This paper proposes a principled framework for constructing structured world models, unifying discrete (logical/symbolic) and continuous (physical/dynamical) stochastic processes as core modeling paradigms. It hierarchically composes Hidden Markov Models (HMMs) with switching Linear Dynamical Systems (sLDS), enforcing a fixed causal structure while optimizing only depth parameters—thereby avoiding combinatorial explosion. The framework integrates Partially Observable Markov Decision Processes (POMDPs) with controllable sLDS and employs an incremental joint learning strategy for structure and parameters. Contribution/Results: Evaluated on multimodal generation and pixel-level planning tasks, the model matches deep neural networks in performance while retaining explicit semantic grounding and full traceability—demonstrating the effectiveness, modularity, and interpretability of the proposed paradigm.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Structured Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The field of world modeling is fragmented, with researchers developing bespoke architectures that rarely build upon each other. We propose a framework that specifies the natural building blocks for structured world models based on the fundamental stochastic processes that any world model must capture: discrete processes (logic, symbols) and continuous processes (physics, dynamics); the world model is then defined by the hierarchical composition of these building blocks. We examine Hidden Markov Models (HMMs) and switching linear dynamical systems (sLDS) as natural building blocks for discrete and continuous modeling--which become partially-observable Markov decision processes (POMDPs) and controlled sLDS when augmented with actions. This modular approach supports both passive modeling (generation, forecasting) and active control (planning, decision-making) within the same architecture. We avoid the combinatorial explosion of traditional structure learning by largely fixing the causal architecture and searching over only four depth parameters. We review practical expressiveness through multimodal generative modeling (passive) and planning from pixels (active), with performance competitive to neural approaches while maintaining interpretability. The core outstanding challenge is scalable joint structure-parameter learning; current methods finesse this by cleverly growing structure and parameters incrementally, but are limited in their scalability. If solved, these natural building blocks could provide foundational infrastructure for world modeling, analogous to how standardized layers enabled progress in deep learning.

Problem

• Proposes fundamental building blocks for structured world models combining discrete and continuous processes
• Addresses combinatorial explosion in structure learning by fixing causal architecture with minimal parameters
• Solves scalable joint structure-parameter learning challenge while maintaining model interpretability

Innovation

• Hierarchical composition of discrete and continuous building blocks
• Fixed causal architecture with four depth parameters
• Modular approach supporting passive and active tasks