ACM SIGSOFT SEN Empirical Software Engineering: Introducing Our New Regular Column

2025-10-02

AI Summary

Empirical Software Engineering (ESE) faces systemic challenges including low reproducibility, limited external validity, difficulties in industrial adoption, and the tacit nature of methodological practices—compounded by a lack of structured platforms for meta-level reflection on research itself. This project establishes SEN-ESE, the first dedicated academic forum systematically addressing meta-level issues in ESE: tacit practices, methodological norms, and interdisciplinary collaboration. Employing a mixed qualitative methodology—including expert interviews, focus groups, surveys, and position papers—it fosters an ongoing, community-driven dialogue. Its core contribution is an open, sustainable platform for critical reflection that significantly enhances research transparency, reproducibility, and practical relevance; moreover, it advances the formalization of ESE methodologies and strengthens their industrial applicability.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Knowledge Representation and Reasoning: Knowledge Engineering
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

From its early foundations in the 1970s, empirical software engineering (ESE) has evolved into a mature research discipline that embraces a plethora of different topics, methodologies, and industrial practices. Despite its remarkable progress, the ESE research field still needs to keep evolving, as new impediments, shortcoming, and technologies emerge. Research reproducibility, limited external validity, subjectivity of reviews, and porting research results to industrial practices are just some examples of the drivers for improvements to ESE research. Additionally, several facets of ESE research are not documented very explicitly, which makes it difficult for newcomers to pick them up. With this new regular ACM SIGSOFT SEN column (SEN-ESE), we introduce a venue for discussing meta-aspects of ESE research, ranging from general topics such as the nature and best practices for replication packages, to more nuanced themes such as statistical methods, interview transcription tools, and publishing interdisciplinary research. Our aim for the column is to be a place where we can regularly spark conversations on ESE topics that might not often be touched upon or are left implicit. Contributions to this column will be grounded in expert interviews, focus groups, surveys, and position pieces, with the goal of encouraging reflection and improvement in how we conduct, communicate, teach, and ultimately improve ESE research. Finally, we invite feedback from the ESE community on challenging, controversial, or underexplored topics, as well as suggestions for voices you would like to hear from. While we cannot promise to act on every idea, we aim to shape this column around the community interests and are grateful for all contributions.

Problem

• Addressing research reproducibility and external validity limitations in software engineering
• Improving communication and documentation of implicit empirical research practices
• Establishing platform for discussing meta-aspects of empirical software engineering research

Innovation

• Establishing a regular column for ESE discussions
• Using expert interviews and focus groups
• Addressing meta-aspects and implicit research topics