FBench: A Flexible Benchmark for CFG-Based What-If Exploration of HPC I/O Patterns

2026-06-29

AI Summary

This work addresses the challenge of efficiently conducting “What-If” I/O performance analysis for large-scale HPC applications, which is hindered by the complex interplay among access patterns, middleware, and file systems. The authors propose FBench, the first flexible I/O benchmarking tool based on context-free grammars (CFGs), capable of generating or replaying I/O traces—captured via Recorder—in real time without modifying application code. FBench supports both POSIX and MPI-IO interfaces and enables configuration-driven exploration through JSON-defined optimization strategies. It faithfully reproduces real-world workloads such as IOR, HACC-IO, FLASH Sedov, and LAMMPS. Evaluations on Lustre reveal that collective I/O write bandwidth can be up to 30× lower than ideal, burst buffers improve non-collective write bandwidth by 1.5×, and performance gains of up to 8× are achievable in LAMMPS scenarios, significantly accelerating I/O optimization studies.

Abstract

The I/O performance of large-scale HPC applications depends on a complex interplay of access patterns, middleware optimizations, and file system configurations. To systematically explore these effects without repeatedly rerunning full applications, we introduce FBench, a flexible and code-transparent benchmarking tool for what-if analysis and I/O performance exploration. FBench leverages context-free grammars (CFGs) derived from Recorder traces to either generate simplified global configuration files for benchmark execution or replay I/O patterns on-the-fly without additional preprocessing. It supports both POSIX and MPI-IO interfaces and allows users to inject optimization hints via JSON configuration files, enabling rapid experimentation with I/O settings without code changes. Our evaluation shows that FBench accurately reproduces I/O behavior for both synthetic and real workloads, capturing access patterns and performance trends across diverse optimizations and file system settings. For IOR and HACC-IO, FBench closely matches scaling behavior and sensitivity to Lustre striping parameters. For FLASH Sedov, it reveals that collective I/O on Lustre can yield up to 30x lower write bandwidth than independent I/O, largely independent of striping, and that switching to a burst buffer file system increases non-collective write bandwidth by about 1.5x without additional tuning. The evaluation with LAMMPS shows that FBench can significantly reduce the time required for what-if analyses and, with simple tuning, enable improvements of up to 8x.

Problem

• HPC I/O
• what-if analysis
• I/O performance
• access patterns
• file system configuration

Innovation

• context-free grammar
• what-if analysis
• I/O benchmarking
• HPC I/O patterns
• code-transparent replay