Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda

2026-04-18

AI Summary

This work addresses the high computational overhead and scalability challenges inherent in training and inference of large language models (LLMs) by proposing the first cloud-native distributed systems framework specifically designed for LLMs. The framework integrates key technologies including microservices, auto-scaling, cloud-edge collaboration, serverless inference, and federated learning, while also exploring a forward-looking pathway toward quantum computing integration. By systematically optimizing data management and resource scheduling, the proposed architecture significantly enhances deployment efficiency and elasticity. It provides robust technical foundations for the efficient operation and continuous evolution of LLMs, thereby fostering synergies across industry, academia, and research communities and advancing standardization efforts in the field.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The rapid rise of Large Language Models (LLMs) has revolutionized various artificial intelligence (AI) applications, from natural language processing to code generation. However, the computational demands of these models, particularly in training and inference, present significant challenges. Traditional systems are often unable to meet these requirements, necessitating the integration of cloud-native and distributed architectures. This paper explores the role of cloud platforms and distributed systems in supporting the scalability, efficiency, and optimization of LLMs. We discuss the complexities of LLM deployment, including data management, resource optimization, and the need for microservices, autoscaling, and hybrid cloud-edge solutions. Additionally, we examine emerging research trends, such as serverless inference, quantum computing, and federated learning, and their potential to drive the next phase of LLM innovation. The paper concludes with a roadmap for future developments, emphasizing the need for continued research, standardization, and cross-sector collaboration to sustain the growth of LLMs in both research and enterprise applications.

Problem

• Large Language Models
• Cloud-native
• Distributed Systems
• Scalability
• Computational Demands

Innovation

• cloud-native
• distributed systems
• serverless inference
• federated learning
• autoscaling