monitoring and observability

Cloud-native systems face fragmented observability and challenging root-cause analysis due to their distributed, highly dynamic architectures. To address this, this paper proposes a reusable, observability-oriented design pattern system comprising three core categories: distributed tracing, application-level metric modeling, and infrastructure-level metric collection. Unlike ad-hoc toolchain integrations, our work is the first to systematically abstract industrial practices into structured, composable design patterns that holistically guide end-to-end monitoring architecture. Implemented and validated using mainstream frameworks—including OpenTelemetry and Prometheus—the approach significantly improves latency attribution accuracy, resource utilization assessment efficiency, and anomaly detection timeliness. Empirical evaluation across multiple microservice deployments demonstrates a 42% reduction in mean time to identify failures.

A key open question in microservice resilience modeling is whether asynchronous semantics—such as Kafka-based message passing—must be explicitly represented in dependency graphs. Method: We propose the first fully automated approach to construct service dependency graphs with asynchronous semantics (e.g., non-blocking Kafka edges) and endpoint success predicates directly from raw OpenTelemetry traces, integrated with closed-loop validation via Monte Carlo simulation and chaos engineering experiments. Contribution/Results: Applied to the OpenTelemetry Demo real-world system, our method achieves end-to-end automation—from trace ingestion to dependency graph construction, availability prediction, and experimental validation—for the first time. Quantitative evaluation shows that incorporating asynchronous semantics has negligible impact (≤10⁻⁵) on instantaneous HTTP endpoint availability predictions; thus, a simple connectivity-based model suffices. This work advances trace-driven resilience modeling from manual, ad-hoc construction toward automation, standardization, and empirical verifiability.

Conventional network telemetry frameworks struggle to support fine-grained traffic measurement, performance diagnostics, and attack detection under stringent memory and computational constraints of high-speed network devices. Method: This paper proposes a lightweight, real-time online telemetry framework that systematically integrates compact data structures—including Bloom filter variants, Count-Min Sketch, and HyperLogLog—with streaming algorithms, hierarchical sampling, and P4-programmable data-plane co-design to comply with hardware limitations. Contribution/Results: Evaluated at line rate exceeding 100 Gbps, the framework reduces memory footprint by over 60% compared to state-of-the-art approaches while maintaining sub-1% flow frequency estimation error. It achieves an optimal trade-off among accuracy, throughput, and resource overhead, thereby significantly enhancing the feasibility and practicality of telemetry in high-bandwidth environments.

To address the infeasibility of manual analysis for large-scale IT system logs, this paper proposes a lightweight log analysis framework leveraging large language models (LLMs). The method introduces a CPU-efficient inference mechanism that significantly improves LLM throughput on resource-constrained hardware without compromising semantic understanding fidelity. It integrates log parsing, contextual modeling, and fault-oriented semantic reasoning to enable end-to-end automated diagnosis. Deployed in production, the system supports 70 software products and has processed over 2,000 incident tickets. Empirical evaluation demonstrates an average monthly reduction of more than 300 human labor hours compared to conventional approaches—equivalent to approximately USD 15,444 in cost savings. The framework thus advances practical, scalable, and cost-effective LLM-based log analytics for real-world operational environments.

This work addresses the operational complexity of 5G/6G O-RAN networks arising from their decoupled architecture and fine-grained control, which hinder the correlation of heterogeneous events and safe generation of configuration actions. The authors propose Net Analyzer rApp, the first framework to integrate a large language model (LLM) as a reasoning collaborator within the O-RAN non-real-time RIC, establishing an event-driven batch inference pipeline for mobility event parsing, anomaly validation, and configuration auditing. By incorporating tool gating, log-directed verification, and human-in-the-loop approval mechanisms, the system strictly decouples reasoning from execution, ensuring auditability and operational safety. Evaluated on a real-world O-RAN testbed under a ping-pong handover scenario, the approach successfully transforms raw telemetry into structured explanations and controlled remediation recommendations, demonstrating both efficacy and security.

This work addresses a critical limitation in existing observability tools for multi-agent systems, which merely log interactions without enabling real-time enforcement of governance policies, thereby allowing violations to be detected only post hoc. To overcome this, the paper proposes a closed-loop governance architecture that embeds governance attributes directly into the telemetry layer, introducing a Governance-aware Telemetry System (GTS). By integrating an OPA-compatible declarative rule engine, a Governance Execution Bus (GEB), and a trusted telemetry plane, the system achieves real-time policy evaluation and tiered intervention with sub-200-millisecond latency. Leveraging encrypted provenance and enriched telemetry data, GTS establishes a tight feedback loop between observation and action, significantly enhancing runtime compliance and security in multi-agent AI systems.

This work proposes an automated log aggregation and analysis framework based on large language models to address the growing challenge of log analysis in increasingly complex systems, where engineers traditionally rely on domain expertise to manually craft intricate LogQL queries. The framework enables end-to-end generation of LogQL queries from natural language instructions by integrating a hierarchical log knowledge base, natural language understanding, knowledge retrieval, and tool invocation mechanisms. Evaluated on four real-world log datasets, the approach achieves an average accuracy of 76.8%, significantly outperforming existing baselines and demonstrating its effectiveness and practicality for log analysis tasks.

This study addresses the challenge of large-scale unsolicited Internet traffic targeting Internet of Things (IoT) devices and its associated security threats by proposing a lightweight monitoring approach that operates without payload inspection. Leveraging data collected via network telescopes, the method integrates privacy-preserving metadata analysis, behavioral heuristics, and Shannon entropy measurements to effectively identify coordinated scanning and backscatter activities. The findings reveal that the top 1% of source IP addresses generate over 81% of the observed traffic, with Telnet ports (23/2323) dominating the activity—evidence of highly concentrated, synchronized, and multi-vector reconnaissance campaigns. This work provides a scalable and practical analytical framework for enhancing large-scale IoT threat situational awareness.

This study addresses the lack of systematic evaluation of mainstream security logging standards in terms of their effectiveness for threat detection. The authors propose a scalable and reproducible assessment methodology based on an automated Security Exploit Telemetry Collection (SETC) framework, which reproduces 50 remote code execution vulnerabilities in containerized environments. Using this approach, they comparatively evaluate the telemetry completeness and attack detectability of widely adopted standards—including Common Information Model (CIM), Open Cybersecurity Schema Framework (OCSF), and Elastic Common Schema (ECS). The experiments quantitatively measure each standard’s detection efficacy, revealing significant disparities in coverage of critical attack indicators and identifying notable gaps. These findings provide empirical guidance for security practitioners in selecting appropriate logging standards to enhance threat detection capabilities.