🤖 AI Summary
To address the cache adaptability deficiency, poor data locality, and escalating storage/network pressure caused by explosive growth in high-energy physics (HEP) data—e.g., from HL-LHC and DUNE—this paper proposes an hourly adaptive caching strategy. Methodologically, it introduces the first HEP-domain file-granularity hourly access predictor and establishes a two-tier machine learning framework integrating LSTM and CatBoostRegressor to enable fine-grained, dynamic data prefetching and intelligent placement. Evaluated on real SoCal MINI cache traces from August 2024, the approach significantly improves cache hit rate and data locality. Furthermore, the WRENCH simulation platform has been extended to support comprehensive evaluation across multi-level heterogeneous systems. Key contributions include: (1) the first hourly file-access prediction model tailored for HEP workloads; (2) a hybrid ML framework balancing temporal dynamics and feature-rich static attributes; and (3) scalable integration into production-grade simulation infrastructure for realistic system-level assessment.
📝 Abstract
Although benefits from caching in US HEP are well-known, current caching strategies are not adaptive i.e. they do not adapt to changing cache access patterns. Newer developments such as High Luminosity - Large Hadron Collider (HL-LHC), Deep Underground Neutrino Experiment (DUNE), a steady move toward streaming readout based Data Acquisition systems (DAQs) will increase the data production exponentially and hence burden the storage, compute &network infrastructures. Moreover, existing caching frameworks are optimized to reduce latency, but not optimized for storage. This in combination with limited cache capacities relative to total data makes it difficult to achieve data locality. In this work, we present Machine Learning-aided (ML) caching strategies. Specifically, first we present a Long Short-Term Memory-based (LSTM) hourly cache usage prediction. Second, we present an hourly file-level access prediction model based on CatboostRegressor. To date, most ML-based cache prediction strategies in HEP have focused on daily cache usage and limited works tackled hourly cache usage and even less strategies addressed hourly file-level access prediction. File-level access prediction allows for the design of intelligent prefetching and data placement strategies with fine-grained control. We validated our cache prediction strategies using data collected from SoCal MINI caches in August 2024. We are currently extending WRENCH simulator to reflect the US HEP ecosystem at the storage, network and compute levels. We plan to deploy our cache prediction strategies into WRENCH and later perform extensive analysis with complex data access patterns and candidate infrastructure configurations.