🤖 AI Summary
This work challenges the conventional practice of treating branch misprediction and cache miss penalties as independent performance bottlenecks, which overlooks their coupling effects under irregular workloads and consequently underestimates the potential of joint optimization. The study is the first to reveal and quantify the phenomenon of symbiotic stall latency (SSL), introduces the Symbiotic Stall Opportunity (SSO) metric to efficiently identify highly coupled workloads, and proposes Joint Speedup Synergy (JSS) to evaluate the benefits of coordinated optimization. Through lightweight MPKI analysis, full-system simulation, and oracle experiments across 53 workloads, the authors find that 70% exhibit significant coupling, with 40% surpassing the theoretical upper bound of independent speedups by over 6%; notably, high-SSO workloads achieve JSS values of 1.23–3.29. These results expose the limitations of traditional decoupled analysis and lead to the identification of four canonical software coupling patterns.
📝 Abstract
Branch mispredictions and data-cache misses are usually evaluated as separate bottlenecks: studies report perfect-branch or perfect-cache speedups as isolated upper bounds and often treat their product as the joint ceiling. In irregular workloads, however, hard-to-predict branches and cache-missing loads often occur in the same hot loops. Removing one penalty can expose the other: faster memory reaches mispredicted branches sooner, while better branch prediction leaves more long-latency loads in the out-of-order window. We call this interaction symbiotic stall latency (SSL).
This paper quantifies when isolated ceilings fail using joint speedup synergy (JSS), the observed joint perfect-branch/perfect-cache speedup divided by the product of the isolated speedups. Values above one mean independent-ceiling analysis understates attainable gain. Across 53 simulated workloads, 70% show measurable coupling (JSS > 1), though many are near unity, especially in lower-pressure cases. With a conservative threshold, 40% exceed the independence product by more than 6%, and kernels with SSO > 20 show JSS from 1.23 to 3.29. We introduce symbiotic stall opportunity (SSO), a lightweight MPKI-based screen for workloads that merit full joint simulation.
We map high-SSO workloads to four recurring software patterns: neighbor access, hash lookup, linked-structure traversal, and data-dependent modification. We connect SSL to reorder-buffer occupancy, squash rate, and commit starvation under isolated perfect modes. The resulting methodology is simple: use SSO to screen, JSS to validate, and report conditional branch-after-cache and cache-after-branch gains when evaluating branch predictors, prefetchers, caches, or coupled branch/memory mechanisms. Our contribution is a measurement framework showing when isolated perfect modes are adequate and when they understate joint performance headroom.