Problem
Research questions and friction points this paper is trying to address.
Generalist Open-World Temporal Perception
multimodal world model
temporal perceptual substrate
structured world state
cross-modal capabilities
Generalist Open-World Temporal Perception
🤖 AI Summary
本文提出一种通用开放式时间感知架构GOWTPA，通过整合多模态数据流来解决AI系统中对生物、物理结构及人工制品的统一表示和理解问题。
📝 Abstract
The next generation of artificial intelligence systems will likely be natively temporal and multimodal in both inputs and outputs: able to converse, perceive, predict, reason, and synthesize through a shared world representation. Realizing this requires a temporal perceptual substrate integrating sensory streams, language, and structured outputs within a multimodal world model. We seek a generalist open-world perceptual system that represents biological forms, natural physical structures, and artifacts, and their interactions, as a coherent, temporally persistent process. The model should infer geometry, articulation, semantics, interaction structure, and uncertainty from raw multimodal streams; maintain identity through occlusion and viewpoint change; generalize across species, forms, mechanisms, and materials; and abstain or expand its ontology when encountering the unknown. The objective is a structured world state supporting understanding, prediction, counterfactual reasoning, and controllable synthesis. Recent work suggests that some cross-modal and reasoning-like capabilities can emerge from large-scale generative video pretraining, reminiscent of language-model scaling. Yet these capabilities are often accessed through language probes or expressed through photorealistic video, leaving explicit semantic, geometric, or temporal structure largely unexposed. This paper articulates an alternative and complementary paradigm: perception and synthesis as distinct conditionings within a shared Generalist Open-World Temporal Perception Architecture (GOWTPA). Recognition, structured prediction, and simulation arise as different conditionings of the same generative substrate, while reasoning and embodiment-specific policies build upon the resulting world state. This positions generalist temporal perception as a potential foundation layer for broader multimodal intelligence and physical AI.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Generalist Open-World Temporal Perception Architecture
multimodal world model
cross-modal capabilities
structured prediction
generative substrate
🔎 Similar Papers
No similar papers found.
