Towards a Small Language Model Lifecycle Framework

2025-06-09

AI Summary

Current research on small language models (SLMs) is fragmented, lacking a unified lifecycle perspective and mechanisms for cross-stage协同 optimization. Method: We propose the first modular, lifecycle-aware framework for SLMs’ full lifecycle, grounded in a systematic survey of 36 works and built via framework engineering—abstracting core, optional, and cross-phase modules while explicitly modeling inter-stage dependencies and synergies. Contribution/Results: The framework enables method reuse, joint adaptation across stages, and stage-coupled optimization, achieving the first structured integration of SLM theoretical research and industrial practice. It delivers an extensible, easily integrable development paradigm and practical guidelines, establishing a unified foundation for SLM design, deployment, maintenance, and toolchain construction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Background: The growing demand for efficient and deployable language models has led to increased interest in Small Language Models (SLMs). However, existing research remains fragmented, lacking a unified lifecycle perspective. Objective: This study aims to define a comprehensive lifecycle framework for SLMs by synthesizing insights from academic literature and practitioner sources. Method: We conducted a comprehensive survey of 36 works, analyzing and categorizing lifecycle-relevant techniques. Results: We propose a modular lifecycle model structured into main, optional, and cross-cutting components. The model captures key interconnections across stages, supporting method reuse, co-adaptation, and lifecycle-awareness. Conclusion: Our framework provides a coherent foundation for developing and maintaining SLMs, bridging theory and practice, and guiding future research and tool development.

Problem

• Define a unified lifecycle framework for Small Language Models (SLMs).
• Synthesize insights from academic and practitioner sources on SLMs.
• Propose a modular model to support method reuse and lifecycle-awareness.

Innovation

• Modular lifecycle model for SLMs
• Method reuse and co-adaptation
• Bridging theory and practice