From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models

2026-03-04

AI Summary

Current large language models are predominantly confined to static reasoning and struggle to operate effectively in dynamic, real-time environments. Moreover, the notion of “streaming large language models” remains ill-defined and lacks a systematic taxonomy. This work presents the first unified definition of streaming large language models, establishing a clear classification framework grounded in data streams and dynamic interaction mechanisms to disambiguate conflated concepts such as streaming generation, streaming input, and interactive architectures. Through a comprehensive literature review and conceptual modeling, the study systematically analyzes existing methodologies, application scenarios, and core technical components, and introduces a continuously updated open-source repository. This foundational effort advances the theoretical underpinnings of streaming intelligence and promotes the field’s progression toward standardization and systematic development.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Data Stream Mining

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Standard Large Language Models (LLMs) are predominantly designed for static inference with pre-defined inputs, which limits their applicability in dynamic, real-time scenarios. To address this gap, the streaming LLM paradigm has emerged. However, existing definitions of streaming LLMs remain fragmented, conflating streaming generation, streaming inputs, and interactive streaming architectures, while a systematic taxonomy is still lacking. This paper provides a comprehensive overview and analysis of streaming LLMs. First, we establish a unified definition of streaming LLMs based on data flow and dynamic interaction to clarify existing ambiguities. Building on this definition, we propose a systematic taxonomy of current streaming LLMs and conduct an in-depth discussion on their underlying methodologies. Furthermore, we explore the applications of streaming LLMs in real-world scenarios and outline promising research directions to support ongoing advances in streaming intelligence. We maintain a continuously updated repository of relevant papers at https://github.com/EIT-NLP/Awesome-Streaming-LLMs.

Problem

• Streaming Large Language Models
• Dynamic Interaction
• Real-time Scenarios
• Systematic Taxonomy
• Static Inference

Innovation

• Streaming Large Language Models
• Dynamic Interaction
• Systematic Taxonomy
• Real-time Inference
• Data Flow