Multi-objective Optimization in CPU Design Space Exploration: Attention is All You Need

2024-10-24

arXiv.org

AI Summary

Addressing three key challenges in CPU microarchitecture design space exploration (DSE)—degraded surrogate model accuracy and scalability, inefficient acquisition strategies, and poor interpretability—this paper proposes AttentionDSE, the first DSE framework integrating attention mechanisms. It unifies high-accuracy performance prediction with real-time bottleneck identification via interpretable, dynamic mapping from architectural parameters to performance contributions. Methodologically, AttentionDSE synergistically combines attention-based modeling, multi-objective optimization, and Pareto frontier search, enabling adaptive analysis under design modifications. Evaluated on the SPEC CPU 2017 benchmark suite, it reduces exploration time by over 80% and improves Pareto hypervolume by 3.9%, while significantly outperforming state-of-the-art approaches in both prediction accuracy and scalability.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Design space exploration (DSE) enables architects to systematically evaluate various design options, guiding decisions on the most suitable configurations to meet specific objectives such as optimizing performance, power, and area. However, the growing complexity of modern CPUs has dramatically increased the number of micro-architectural parameters and expanded the overall design space, making DSE more challenging and time-consuming. Existing DSE frameworks struggle in large-scale design spaces due to inaccurate models and limited insights into parameter impact, hindering efficient identification of optimal micro-architectures within tight timeframes. In this work, we introduce AttentionDSE. Its key idea is to use the attention mechanism to establish a direct mapping of micro-architectural parameters to their contributions to predicted performance. This approach enhances both the prediction accuracy and interpretability of the performance model. Furthermore, the weights are dynamically adjusted, enabling the model to respond to design changes and effectively pinpoint the key micro-architectural parameters/components responsible for performance bottlenecks. Thus, AttentionDSE accurately, purposefully, and rapidly discovers optimal designs. Experiments on SPEC 2017 demonstrate that AttentionDSE significantly reduces exploration time by over 80% and achieves 3.9% improvement in Pareto Hypervolume compared to state-of-the-art DSE frameworks while maintaining superior prediction accuracy and efficiency with an increasing number of parameters.

Problem

• Addresses poor scalability of surrogate models in CPU design space exploration
• Improves inefficient acquisition methods in high-dimensional architectural spaces
• Enhances interpretability to identify architectural bottlenecks in CPU design

Innovation

• Attention-based neural architecture for DSE
• Perception-Driven Attention with O(n) complexity
• Attention-aware Bottleneck Analysis for optimization