model training and lifecycle

In large-scale DNN distributed training, checkpointing is tightly coupled with model parallelism strategies and hardware topology, severely limiting fault tolerance and elastic scalability. To address this, we propose the “distributed storage, unified loading” paradigm: during saving, model parameters are stored in a distributed representation aligned with the current parallel configuration; during restoration, they are uniformly reconstructed into a logically consistent parameter view. We design a universal checkpoint format—incorporating merged parameter representations and mapping metadata—a Universal Checkpoint Language (UCL), and an on-demand state reconstruction mechanism, achieving, for the first time, full decoupling of checkpointing from parallel configurations. Evaluated on LLaMA, Bloom, and other mainstream large models under diverse parallelism paradigms—including tensor parallelism (TP), pipeline parallelism (PP), data parallelism (DP), and context parallelism (CP)—our approach reduces post-failure recovery time by 12–28% on average, significantly enhancing cross-configuration portability and system robustness.

Current MCP systems rely solely on semantic matching, rendering them vulnerable to network latency fluctuations and server failures, thereby compromising the robustness of LLM-based tool invocation. To address this, we propose NetMCP—the first network-aware MCP experimental platform—that jointly optimizes semantic matching and network quality through real-time monitoring of network QoS and server health. Its core innovation is the SONAR routing algorithm, which unifies semantic similarity, latency prediction, and service availability modeling into a single dynamic routing decision framework—a novel integration in MCP design. NetMCP supports heterogeneous service integration and latency-sequence-driven stress testing. Experimental results demonstrate that, compared to semantic-only and LLM-based baselines, SONAR improves task success rate by 23.6%, reduces average completion time by 31.4%, and decreases failure count by 47.2%.

To address interdisciplinary interoperability, variant configuration governance, end-to-end traceability, and cross-organizational collaboration challenges arising from the networked evolution of Systems of Systems (SoS), this paper proposes a lifecycle management framework for Network-Centric Development (NCD). Methodologically, it grounds the framework in Model-Based Systems Engineering (MBSE) semantics and integrates Product Lifecycle Management (PLM) governance, CAD-CAE model synchronization, and closed-loop digital thread/digital twin capabilities. Its core contributions are four foundational principles: (1) reference architecture with a unified data model; (2) end-to-end configuration sovereignty; (3) review-driven model gating; and (4) quantifiable value contribution assessment. Empirical validation across transportation, healthcare, and public-sector domains demonstrates significant improvements in change robustness and model reuse rate, reduced delivery cycles, and enhanced support for sustainability-oriented decision-making.

Training large foundation models (LFMs) faces significant challenges in checkpoint management, including poor cross-framework compatibility, tight coupling with parallelization strategies, heterogeneous storage backends, and severe I/O bottlenecks. To address these, this work proposes an industrial-grade unified archival system. Its core contributions are: (1) a novel parallelism-agnostic checkpoint serialization format; (2) a full-stack I/O optimization framework integrating a dynamic resharding engine, multi-framework abstraction interfaces (PyTorch/Megatron/DeepSpeed), asynchronous high-throughput storage adapters, and a distributed I/O monitoring toolchain; and (3) runtime support for cross-parallelism resharding, multi-backend adaptivity, and rapid failure recovery. Experiments demonstrate an average 54.20× reduction in checkpoint blocking time, with peak checkpoint save and load speedups of 9.96× and 8.80×, respectively. The system has been stably deployed in production environments scaling to over one thousand GPUs.

This work presents the first comprehensive synthesis of the technical foundations of the Meta Llama 4 model family, addressing the current lack of systematic documentation. It details core architectural innovations—including the routed/shared mixture-of-experts design, early-fusion multimodal integration, iRoPE-based long-context extension, and lightweight alignment strategies such as light supervised fine-tuning (SFT), online reinforcement learning (RL), and light direct preference optimization (DPO). The study integrates the complete training pipeline, evaluation results, and deployment constraints, offering an authoritative technical reference. Furthermore, it compiles performance benchmarks for both base and instruction-tuned variants across standard datasets and clarifies practical considerations for inference, including context-length limitations and quantization-aware deployment practices.

This work addresses the substantial storage and serving overhead incurred by fine-tuning large language models with massive numbers of LoRA adapters by introducing the MindLab Toolkit (MinT), a hosting framework that shares a common base model while transmitting only lightweight LoRA adapters and managing their full lifecycle uniformly. Key innovations include support for catalogs of up to one million LoRA strategies, adapter compression to less than 1% of the base model size, decoupling of persistent storage from computational address spaces, and integration of tensor parallelism, GRPO-based concurrent multi-strategy training, batched MoE-LoRA loading, and cold-start-aware scheduling. Experiments demonstrate an 18.3× inference speedup on a 4B dense model and a 2.85× speedup on a 30B MoE model; a single engine can traverse 100,000 strategies, clusters support over a thousand concurrent requests, and MoE loading efficiency improves by 8.5–8.7×.

This work addresses the growing complexity of CI/CD pipelines and the lack of structured analysis capabilities in existing tools for understanding their behavior, failures, and version evolution. The authors propose an innovative approach that uniquely integrates digital twin technology with BPMN-based modeling in DevOps contexts. By automatically parsing raw CI configurations and execution logs, the method constructs structured, high-level process models that enable pipeline visualization, failure traceability, and cross-version comparison. Evaluated across multiple open-source projects, the approach demonstrates effectiveness in monitoring, evolutionary analysis, and fault diagnosis, offering a modular and extensible foundational framework for the analysis and optimization of CI/CD pipelines.

This work addresses the lack of structured, verifiable, and governable tool support for large language model (LLM) agents in operational tasks, where existing approaches are often static or manually integrated, struggling to balance security and extensibility. The authors propose the “Tool Capsule” paradigm, which encapsulates tools as standardized units comprising intent, contract, implementation, policy, and verification evidence. They design an efficient intent-scoped routing mechanism enabling on-demand, secure tool invocation. The system integrates a sandboxed verification pipeline, MCP-compatible routing, credential binding, and lifecycle governance. Experiments demonstrate a micro F1 score of 0.901 across 83 routing tests with a 99.2% reduction in context overhead; all 25 end-to-end tasks produced valid toolkits (micro F1 = 0.940), with 23 successfully passing real-time sandbox validation.

Existing large-scale model training systems struggle to flexibly compose diverse parallelization strategies, often relying on manual expert tuning and lacking generality. This work proposes a programmable distributed training system that enables users to declaratively specify composite parallelism strategies—such as data, pipeline, and expert parallelism—through model annotations and scheduling directives. These specifications are compiled via a unified intermediate representation (IR) into device-level execution plans, fully decoupling strategy definition from runtime execution over a global compute-communication DAG. The system is the first to support automatic compilation of user-defined composite strategies, matching the performance of established approaches like ZeRO while significantly improving both performance and memory efficiency in complex scenarios such as DeepSeek-V3’s DualPipe.

This study addresses the challenges of poor interoperability among cross-domain distributed cyber ranges and insufficient automation in evaluation within cyber defense training. To overcome these limitations, the authors propose the ACTING platform, which introduces the Exercise Description Language – First Generation (EDL-FG), a novel scenario description language enabling joint deployment and coordinated exercises across multiple ranges. Built upon a federated architecture, ACTING integrates a unified performance assessment framework with automated data collection mechanisms, facilitating structured definition of training scenarios, cross-domain automated orchestration, and quantitative scoring. This work significantly enhances the interoperability, scalability, and operational realism of cyber attack–defense exercises, thereby providing an effective foundation for multi-domain collaborative training in civil-military integration contexts.