A Methodology for Investigating AI Patterns Prevalence in Software Repositories

2026-07-01

AI Summary

This study addresses the lack of empirical validation regarding the practical application of AI design patterns in real-world codebases. By mining the literature, the authors identify 14 distinct AI design patterns and, for the first time, integrate active learning with pattern mining to construct a quantifiable analytical framework across 100 open-source AI projects on GitHub. This framework enables estimation of the occurrence frequency and statistical confidence bounds for each pattern. Evaluated on an 8-class classification task, the approach achieves 56% accuracy and 55% recall—substantially outperforming the 11% random baseline. The work fills a critical gap in empirical research on AI design patterns and provides practitioners with actionable estimates of pattern applicability boundaries.

Abstract

As Artificial Intelligence(AI)-based applications take off, a clear understanding of AI patterns can uplift the quality of AI applications. Many AI patterns have been proposed in the literature; however, their prevalence in real-life code has not yet been validated. Understanding the actual use of those patterns in practice can clarify our understanding both of the significance of these patterns and their utility. In this paper, we present a methodology to a) identify relevant patterns by mining the literature and then to b) validate their presence and prevalence in actual code repositories using active learning. To that end, we identify 14 AI pattern classes by mining 44 published AI pattern-related sources. Then we use an active learning approach to determine the prevalence of the most common pattern class across 100 GitHub open AI repositories. Using prevalence estimation, we propose bounds on the accuracy of the occurrences. The model achieves 56\% accuracy and 55\% recall in an 8-way classification task, significantly outperforming the 11\% random-chance baseline. Furthermore, the prevalence estimation offers usable bounds for analyzing pattern applications. This methodology provides a robust foundation to start understanding how AI patterns are used in practice, a field that currently lacks empirical data.

Problem

• AI patterns
• prevalence
• software repositories
• empirical validation
• real-world usage

Innovation

• AI patterns
• active learning
• prevalence estimation
• software repositories
• empirical validation