software development

Implementing and delivering software features using languages, frameworks, version control (Git), build systems, IDEs, and developer workflows; involves writing modular code, running debuggers, setting up CI/CD pipelines, participating in code review, and using package managers and dependency management.

softwaredevelopment

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the lack of systematic understanding regarding the evolution of GitHub Actions workflows. Through a mixed-methods approach, we conduct the first large-scale empirical analysis of over 3.4 million workflow file versions from more than 49,000 repositories spanning November 2019 to August 2025. We identify seven categories of conceptual changes and find that repositories typically contain a median of three workflow files, with 7.3% of workflows modified weekly—approximately 75% of which involve only a single change, predominantly in task configuration and specification. Our findings further indicate that current large language model (LLM) tools have not yet significantly influenced workflow maintenance frequency, offering empirical grounding for the design of fine-grained automated maintenance tools.

CI/CDempirical studyGitHub Actions

What is a Feature, Really? Toward a Unified Understanding Across SE Disciplines

Feb 14, 2025
NP
Nitish Patkar
🏛️ University of Applied Sciences and Arts Northwestern Switzerland (FHNW) | University of Fribourg | University of Bern

Inconsistent definitions of “feature” across software engineering domains—particularly requirements engineering (RE) and software product lines (SPL)—impede communication, trigger rework, and reduce cross-team collaboration efficiency. Method: We conducted an empirical study across 27 mainstream open-source projects, integrating repository mining, branch behavior analysis, qualitative coding, and pattern induction to derive a data-driven, cross-disciplinary definition of feature. Contribution/Results: This work introduces the first empirically grounded, unified feature definition framework bridging RE and SPL. It identifies recurring collaboration patterns and critical bottlenecks in feature description, implementation, and management, and proposes a roadmap linking academic theory with industrial practice. The findings yield actionable guidelines for project planning, resource allocation, and inter-team coordination, advancing feature conceptual standardization and engineering practice optimization.

Address communication gaps and inefficienciesImprove project planning and coordinationUnify feature understanding across SE disciplines

This study investigates the feasibility of leveraging large language models (LLMs) to automatically generate GitHub Actions CI configurations, addressing the labor-intensive and error-prone nature of manual YAML authoring. We introduce the first annotated CI configuration dataset derived systematically from official documentation and conduct a comprehensive zero-shot evaluation of six state-of-the-art models—including GPT-4o, Llama, and Gemma—on the natural-language-to-YAML generation task. Results show that general-purpose LLMs significantly outperform code-specialized models; the highest structural similarity reaches 69%, yet only 3% of generated configurations are fully executable. Primary failure modes include missing workflow steps and semantic misinterpretations of CI semantics. The study uncovers a critical gap between semantic understanding and structural precision in current LLMs for CI configuration generation, establishing a new benchmark and empirical foundation for evaluating and enhancing LLM capabilities in DevOps-oriented automation.

Assessing accuracy and limitations of LLMs in YAML-based CI tasksEvaluating LLMs for generating GitHub Actions CI configurationsIdentifying gaps in LLM-generated CI configurations' structural correctness

Exploringand Unleashing the Power of Large Language Models in CI/CD Configuration Translation

Nov 03, 2025
JW
Jiajun Wu
🏛️ Soochow University | Nanyang Technological University | University of Shanghai for Science and Technology

This work addresses the challenge of automated configuration migration across CI platforms—particularly from Travis CI to GitHub Actions—where manual translation is error-prone and labor-intensive. We propose an LLM-based translation framework grounded in empirical analysis of 811 real-world migration cases, enabling the first quantitative characterization of configuration conversion effort. We introduce a four-category taxonomy of translation problems and identify recurring developer pain points. Methodologically, we design a composite prompting strategy integrating documentation-guided instruction, iterative refinement, and in-context learning to enhance LLM robustness and fidelity. Evaluation on GPT-4o shows our approach achieves 75.5% end-to-end build success rate—nearly tripling the performance of baseline prompting—while substantially reducing manual intervention. The framework provides a reproducible, quantitatively evaluable pathway for intelligent CI/CD configuration migration.

Addressing semantic differences in CI migration processesImproving accuracy of automated CI configuration translationTranslating CI configurations between different platforms

Empirical Analysis on CI/CD Pipeline Evolution in Machine Learning Projects

Mar 18, 2024
AH
Alaa Houerbi
🏛️ University of Michigan- Dearborn

This study presents the first empirical investigation into the evolution of CI/CD configurations in machine learning (ML) projects. Addressing the lack of understanding regarding how CI/CD configurations co-evolve with ML components, the authors analyze 508 open-source ML projects, 343 manually annotated commits, and 15,634 automated CI/CD commits. They propose a novel 14-category taxonomy capturing synergistic changes between CI/CD and ML components, develop a dedicated clustering tool to identify recurrent evolutionary patterns, and establish an empirically grounded model linking developer experience to CI/CD configuration modification behavior. Results show that 61.8% of CI/CD-related commits involve build strategy modifications; common anti-patterns—including dependency hardcoding and missing test frameworks—are identified; and senior developers modify CI/CD configurations more frequently and effectively than juniors, confirming the critical role of experience in CI/CD maintenance.

Analyzes CI/CD evolution in ML projectsDevelops clustering tool for CI/CD patternsIdentifies common CI/CD configuration changes

Latest Papers

What's happening recently
View more

This study addresses the lack of systematic understanding in the configuration and maintenance of CI/CD caching, which imposes a significant burden on developers despite its benefits for build efficiency. Through a large-scale empirical analysis of 952 repositories on GitHub Actions—encompassing 1,556 workflow files and over ten thousand cache-related changes—the authors employ code mining, configuration analysis, commit tracing, and statistical modeling to uncover real-world caching practices, evolutionary patterns, and human-bot collaboration in maintenance. The findings reveal that cache adopters are more active, caching strategies are diverse and frequently adjusted, and build- and test-related tasks evolve rapidly. Manual interventions primarily address misconfigurations, whereas version upgrades are predominantly automated by bots. The work quantifies the maintenance overhead of caching and provides empirical foundations for improving developer tooling.

cache maintenanceCI/CD cachingempirical study

This study addresses the lack of systematic understanding regarding how GitHub Actions workflows are used in real-world scenarios, how developers respond to workflow failures, and how these practices relate to project characteristics. Combining large-scale quantitative analysis of 258,300 workflow runs with qualitative case studies across 21 diverse repositories, this work identifies three typical patterns developers employ to handle workflow failures and uncovers a “configuration–usage gap”—where YAML configurations exist but workflows remain effectively unused. Furthermore, the study empirically validates five hypotheses linking project features to workflow usage intensity, revealing a significant positive correlation between high usage intensity and low failure rates. These findings provide actionable empirical evidence for improving CI/CD practices.

CI/CDfailure responseGitHub Actions

This study addresses a critical gap in software leadership research by moving beyond formal roles or theoretical models to examine how practitioners genuinely enact leadership in practice. Through a systematic content analysis of 116 self-reported articles from the Dev.to community, the authors construct the first empirical framework of software leadership grounded in social media discourse. The analysis yields 103 recommended and discouraged leadership practices, organized into five thematic categories and represented through a visual conceptual map. Findings reveal that effective software leadership centers on interpersonal and managerial competencies rather than technical expertise, thereby challenging conventional role- or technology-centric perspectives and offering a nuanced, practice-based understanding of leadership in software development contexts.

leadership practicespractitioner experiencequalitative analysis

This work addresses the fragility, inefficiency, and strong platform coupling commonly found in CI/CD pipelines for legacy COBOL systems, which often result in high maintenance costs and vendor lock-in. To overcome these challenges, the authors propose a portable CI/CD architecture tailored for highly secure and compliance-driven environments. The approach leverages OCI-compliant container images preloaded with COBOL toolchains, introduces a platform abstraction layer, integrates multiple repositories, and employs Groovy script refactoring to achieve platform-agnostic continuous integration and delivery. Empirical evaluation demonstrates that the proposed solution significantly enhances efficiency—reducing pipeline execution time by 82%—while simultaneously improving system portability, security, and maintainability. This architecture offers a reusable paradigm for modernizing legacy COBOL applications within regulated domains.

CI/CDCOBOLlegacy systems

This work addresses the challenges of IDE development posed by the rapid evolution of smart contract languages such as Move by presenting a high-performance IDE support system built atop the Move compiler and adhering to the Language Server Protocol (LSP). Through deep integration with existing language toolchains and the application of incremental parsing and optimized semantic analysis techniques, the system efficiently delivers rich IDE features even as the language undergoes continuous iteration. Deployed successfully within the Sui platform’s Move ecosystem, it significantly enhances developer experience and yields a reusable, evolution-aware IDE construction strategy applicable to other emerging programming language ecosystems.

evolving language ecosystemIDELanguage Server Protocol

Hot Scholars

CT

Christoph Treude

Associate Professor of Computer Science, Singapore Management University
Software EngineeringEmpirical Software EngineeringHuman-AI InteractionAI for Science
PL

Peng Liang

School of Computer Science, Wuhan University
Software EngineeringSoftware ArchitectureEmpirical Software Engineering
RD

Ronnie de Souza Santos

Assistant Professor, University of Calgary
Human Aspects of Software EngineeringSoftware TestingSoftware FairnessSoftware Development
ZJ

Zhi Jin

Sun Yat-Sen University, Associate Professor
AE

Ahmed E. Hassan

Mustafa Prize Laureate, ACM/IEEE/NSERC Steacie Fellow, ACM Influential/IEEE Distinguished Educator
Mining Software RepositoriesSoftware AnalyticsEmpirical Software EngineeringSoftware