CREDIT: Cost-guided Reduction-reuse with Efficient DSMEM Inter-CTA Tiling

2026-09-01

AI Summary

为解决DSMEM性能优化难题，提出CREDIT框架，通过工作负载模式识别、转换及成本模型预测，实现跨多种工作负载的稳定加速。

Abstract

NVIDIA distributed shared memory (DSMEM) enables direct shared-memory access within a thread block cluster. However, cluster synchronization, remote access, and resource costs make it difficult to determine when DSMEM improves performance. To fill this gap, we propose CREDIT, a cost-guided framework that identifies DSMEM-profitable workload patterns, predicts their profitability range, and delivers consistent speedups across diverse workloads. CREDIT combines three innovations: (1) a profiling-driven characterization that identifies workload patterns likely to benefit from DSMEM; (2) a transformation that applies DSMEM to reduction-reuse workloads; (3) a cost model based on profiling data, to determine its profitability range. Evaluations on diverse workloads show CREDIT achieves 91.7% prediction accuracy on profitability. CREDIT beats torch.compile, Triton, and optimized non-DSMEM CUDA baselines on all six workloads, with geometric-mean speedups of 1.466x on RTX 5090 and 1.318x on H100. CREDIT's source code is publicly available at https://github.com/zhengxiongli08/CREDIT.

Problem

• DSMEM
• performance improvement
• cluster synchronization
• resource costs

Innovation

• cost-guided framework
• workload patterns
• DSMEM
• reduction-reuse
• profitability prediction