GenAI for Systems: Recurring Challenges and Design Principles from Software to Silicon

2026-02-16

AI Summary

Current research on generative AI applications across computing system layers—spanning software, architecture, and chips—is highly fragmented, lacking a unified methodology to address cross-layer challenges. This work presents the first full-stack analysis of generative AI’s role in eleven domains, including code generation, runtime optimization, and hardware design, synthesizing insights from over 275 publications and cross-layer case studies. It identifies five recurring core challenges and distills five corresponding general design principles. By establishing a structured “challenge–principle” mapping framework, this study offers actionable guidance for diagnosing and designing generative AI systems, thereby advancing the development of shared terminology, cross-layer benchmarks, and a systematic engineering methodology for the field.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Neural Architectures and Foundation Models
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Generative AI is reshaping how computing systems are designed, optimized, and built, yet research remains fragmented across software, architecture, and chip design communities. This paper takes a cross-stack perspective, examining how generative models are being applied from code generation and distributed runtimes through hardware design space exploration to RTL synthesis, physical layout, and verification. Rather than reviewing each layer in isolation, we analyze how the same structural difficulties and effective responses recur across the stack. Our central finding is one of convergence. Despite the diversity of domains and tools, the field keeps encountering five recurring challenges (the feedback loop crisis, the tacit knowledge problem, trust and validation, co-design across boundaries, and the shift from determinism to dynamism) and keeps arriving at five design principles that independently emerge as effective responses (embracing hybrid approaches, designing for continuous feedback, separating concerns by role, matching methods to problem structure, and building on decades of systems knowledge). We organize these into a challenge--principle map that serves as a diagnostic and design aid, showing which principles have proven effective for which challenges across layers. Through concrete cross-stack examples, we show how systems navigate this map as they mature, and argue that the field needs shared engineering methodology, including common vocabularies, cross-layer benchmarks, and systematic design practices, so that progress compounds across communities rather than being rediscovered in each one. Our analysis covers more than 275 papers spanning eleven application areas across three layers of the computing stack, and distills open research questions that become visible only from a cross-layer vantage point.

Problem

• Generative AI
• cross-stack
• recurring challenges
• design principles
• systems engineering

Innovation

• cross-stack
• generative AI
• design principles
• recurring challenges
• systems co-design