Beyond Moore's Law: Harnessing the Redshift of Generative AI with Effective Hardware-Software Co-Design

2025-04-09

AI Summary

Moore’s Law’s slowdown undermines the traditional hardware-software abstraction boundary, while existing co-design methodologies fail to meet generative AI’s escalating computational demands and suffer from the “hardware lottery”—an empirical phenomenon where architectural innovations are arbitrarily constrained by hardware availability. Method: We propose a five-stage evolutionary model for hardware-software co-design, integrating computer architecture, ML systems, domain-specific architectures (DSAs), and co-engineering principles. We systematically critique and quantitatively analyze the hardware lottery’s suppressive effect on architectural innovation, and elevate system abstraction to a first-class design concern—redefining cross-layer abstraction hierarchies and design principles. Contribution/Results: Our approach significantly improves co-design efficiency and democratizes architectural innovation. It establishes a scalable, reusable co-design methodology tailored for the generative AI era, thereby embedding architecture research deeply into the core of ML system development.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

For decades, Moore's Law has served as a steadfast pillar in computer architecture and system design, promoting a clear abstraction between hardware and software. This traditional Moore's computing paradigm has deepened the rift between the two, enabling software developers to achieve near-exponential performance gains often without needing to delve deeply into hardware-specific optimizations. Yet today, Moore's Law -- with its once relentless performance gains now diminished to incremental improvements -- faces inevitable physical barriers. This stagnation necessitates a reevaluation of the conventional system design philosophy. The traditional decoupled system design philosophy, which maintains strict abstractions between hardware and software, is increasingly obsolete. The once-clear boundary between software and hardware is rapidly dissolving, replaced by co-design. It is imperative for the computing community to intensify its commitment to hardware-software co-design, elevating system abstractions to first-class citizens and reimagining design principles to satisfy the insatiable appetite of modern computing. Hardware-software co-design is not a recent innovation. To illustrate its historical evolution, I classify its development into five relatively distinct ``epochs''. This post also highlights the growing influence of the architecture community in interdisciplinary teams -- particularly alongside ML researchers -- and explores why current co-design paradigms are struggling in today's computing landscape. Additionally, I will examine the concept of the ``hardware lottery'' and explore directions to mitigate its constraining influence on the next era of computing innovation.

Problem

• Addressing stagnation of Moore's Law via hardware-software co-design
• Reevaluating traditional decoupled system design for modern computing
• Mitigating hardware lottery's impact on future computing innovation

Innovation

• Hardware-software co-design for AI performance
• Elevating system abstractions to first-class citizens
• Mitigating hardware lottery constraints in computing