Scrutinizing Variables for Checkpoint Using Automatic Differentiation

2026-02-17

AI Summary

Traditional checkpointing mechanisms indiscriminately save all data, leading to significant waste of storage and computational resources. This work proposes a fine-grained influence analysis method based on automatic differentiation (AD), which, for the first time, leverages AD to identify—on an element-wise basis—the data elements that critically affect program outputs. By checkpointing only these critical elements, the approach substantially reduces overhead. The method integrates variable-level influence analysis with the NAS Parallel Benchmark (NPB) suite to effectively pinpoint critical regions in scientific applications. Experimental results demonstrate that this strategy achieves up to a 20% reduction in checkpoint storage costs while significantly improving fault-tolerance efficiency.

Technology Category

• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Checkpoint/Restart (C/R) saves the running state of the programs periodically, which consumes considerable system resources. We observe that not every piece of data is involved in the computation in typical HPC applications; such unused data should be excluded from checkpointing for better storage/compute efficiency. To find out, we propose a systematic approach that leverages automatic differentiation (AD) to scrutinize every element within variables (e.g., arrays) for checkpointing allowing us to identify critical/uncritical elements and eliminate uncritical elements from checkpointing. Specifically, we inspect every single element within a variable for checkpointing with an AD tool to determine whether the element has an impact on the application output or not. We empirically validate our approach with eight benchmarks from the NAS Parallel Benchmark (NPB) suite. We successfully visualize critical/uncritical elements/regions within a variable with respect to its impact (yes or no) on the application output. We find patterns/distributions of critical/uncritical elements/regions quite interesting and follow the physical formulation/logic of the algorithm.The evaluation on NPB benchmarks shows that our approach saves storage for checkpointing by up to 20%.

Problem

• Checkpoint/Restart
• High Performance Computing
• Variable Scrutiny
• Storage Efficiency
• Application Output Impact

Innovation

• automatic differentiation
• checkpointing
• HPC
• data relevance analysis
• storage optimization