A Microservice Graph Generator with Production Characteristics

2024-12-26

AI Summary

In microservice architectures, large-scale production data and dynamically evolving invocation patterns—such as interface evolution, cross-service reuse, and high-frequency repetitive calls—hinder the construction of high-fidelity dependency graphs for resource optimization. To address this, we propose DGG (Dependency Graph Generator): it first extracts fine-grained invocation graphs from production traces; then performs hierarchical clustering guided by topology and call semantics, integrating dynamic interface behavior and repetition patterns; finally, synthesizes lightweight, scalable dependency graphs preserving real-world production characteristics via a stochastic graph model. DGG is the first approach to faithfully embed three critical production features—dynamic interfaces, cross-graph sharing, and repetitive invocations—into synthetic graphs, introducing a novel “data organization → category-driven modeling → multi-graph fusion” generation paradigm. Experiments show that DGG-generated graphs significantly outperform baselines in topological fidelity and enable resource scaling policies that improve resource efficiency by 44.8% while maintaining QoS.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

A production microservice application may provide multiple services, queries of a service may have different call graphs, and a microservice may be shared across call graphs. It is challenging to improve the resource efficiency of such complex applications without proper benchmarks, while production traces are too large to be used in experiments. To this end, we propose a Service Dependency Graph Generator (DGG) that comprises a Data Handler and a Graph Generator, for generating the service dependency graphs of benchmarks that incorporate production-level characteristics from traces. The data handler first constructs fine-grained call graphs with dynamic interface and repeated calling features from the trace and merges them into dependency graphs, and then clusters them into different categories based on the topological and invocation types. Taking the organized data and the selected category, the graph generator simulates the process of real microservices invoking downstream microservices using a random graph model, generates multiple call graphs, and merges the call graphs to form the small-scale service dependency graph with production-level characteristics. Case studies show that DGG's generated graphs are similar to real traces in terms of topologies. Moreover, the resource scaling based on DGG's fine-grained call graph constructing increases the resource efficiency by up to 44.8% while ensuring the required QoS.

Problem

• Microservices Architecture
• Resource Efficiency
• Dependency Mapping

Innovation

• Service Dependency Graph Generator
• Microservices Optimization
• Resource Efficiency