A Pilot Study on Detecting Software Design Patterns with Large Language Models: An Empirical Evaluation

2026-04-19

AI Summary

This study addresses the challenge of automatically detecting software design patterns in source code to support architectural understanding and quality assessment. It presents the first systematic evaluation of four large language models—including NextCoder and Gemma 3—as well as two ensemble strategies combining three models, for recognizing five classic design patterns: Singleton, Adapter, Bridge, Composite, and Decorator. The work investigates the impact of three input modalities—raw source code, PlantUML diagrams, and textual descriptions—on detection performance. Experimental results demonstrate that NextCoder and Gemma 3 achieve the highest accuracy among individual models, while ensemble approaches further enhance performance, thereby confirming the effectiveness and potential of large language models in design pattern recognition tasks.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Design patterns provide reusable solutions to recurring software design problems. Automatically detecting these patterns in source code can help bootstrap new developers' understanding of unfamiliar software system architectures, and can help experienced developers to quickly identify and rectify potential quality issues. While many prior research works have explored graph based and machine-learning based detection techniques, this work evaluates the design pattern recognition capabilities of four Large Language Models and two ensemble approaches consisting three out of the four models. We also compare their performance when prompted with a) Source code, b) PlantUML representation of source code, and c) Text-based descriptions of the source code. We investigate the detection of five design patterns: singleton, adapter, bridge, composite and decorator. Our preliminary results indicate that LLMs show promise for automatically detecting design patterns, with NextCoder and Gemma 3 demonstrating comparatively higher accuracy than other models evaluated, and the ensemble approaches enhancing the overall efficiency of design pattern detection. We identify several directions for future work.

Problem

• software design patterns
• automatic detection
• source code analysis
• design pattern recognition
• software architecture understanding

Innovation

• Large Language Models
• Design Pattern Detection
• Ensemble Methods
• Empirical Evaluation
• Software Architecture Understanding