Literature Study on Operational Data Analytics Frameworks in Large-scale Computing Infrastructures

2026-03-19

AI Summary

This study addresses the growing complexity of operating large-scale computing infrastructures, such as high-performance computing (HPC) systems, where existing operational data analytics (ODA) frameworks struggle to effectively support multi-layered, distributed graph processing ecosystems. The work provides a systematic review of core ODA components, benchmarks prevailing approaches, and proposes a novel holistic ODA framework that integrates the distributed graph processing hierarchy introduced by Sherif Sak et al., thereby extending the functional scope of Netti et al.’s prior work. By unifying fine-grained monitoring, ODA architecture, and graph processing system design, the proposed framework significantly enhances structural integrity and functional extensibility, markedly improving operational efficiency. Furthermore, it illuminates key research directions for ODA in high-performance computing environments.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Search and Optimization: Distributed Search
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

By 2025, there are zettabytes of data generated every year. The size and complexity of modern large-scale computing infrastructures like High-Performance Computing (HPC) systems continue to evolve and become complex, leaving us wondering about their manageability and sustainability concerns. Because of this reason, those complex systems are provided with fine-grained monitoring and Operational Data Analytics (ODA) capabilities to optimise their efficiency. In this literature study, we list the fundamental pillars of the large-scale computing infrastructures which enable its ODA capabilities, and conduct a study of the popular ODA frameworks operating in various such environments (predominantly HPC). Based on that, we propose a more holistic ODA framework matching the various layers of a large-scale graph-processing distributed ecosystem proposed by Sherif Sak et al, that extends the ODA functionalities presented in an existing novel ODA framework proposed by Netti et al. We compare the holistic ODA framework proposed by us to some of the state-of-the-art frameworks that we study as part of this literature to highlight the novelty, which would hopefully draw more attention to perform extensive research in this field. As part of creating awareness, we highlight the significant operational efficiencies observed as a result of the implementation of the state-of-the-art ODA frameworks to make the study appear beneficial for the readers, and lastly, discuss the trending research work ongoing in this field.

Problem

• Operational Data Analytics
• Large-scale Computing Infrastructures
• High-Performance Computing
• System Manageability
• Sustainability

Innovation

• Operational Data Analytics
• Holistic Framework
• Large-scale Computing
• HPC Monitoring
• Distributed Graph Processing