An instrument to measure factors that constitute the socio-technical context of testing experience

2025-05-02

AI Summary

In software testing practice, socio-technical factors—such as tools, test cases, collaboration patterns, developer motivation, and mental models—are notoriously difficult to quantify. Method: Grounded in socio-technical systems theory and psychometric scale development principles, this paper employs grounded theory and empirical research to design a structured questionnaire, yielding the first human–technology co-analytical framework for deconstructing “testing experience.” The resulting multidimensional measurement scale demonstrates initial reliability and validity. Contribution/Results: This work pioneers the operationalization of testing experience as a measurable, cross-dimensional construct and explicitly articulates theoretical hypotheses regarding interrelationships among its key factors. The validated scale establishes a reusable methodological foundation and measurement benchmark for large-scale empirical studies and evidence-based optimization of testing practices.

Technology Category

• Search and Optimization: Evaluation and Analysis
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Web Mining and Content Analysis: Web measurements

Abstract

We consider testing a cooperative and social practice that is shaped by the tools developers use, the tests they write, and their mindsets and human needs. This work is one part of a project that explores the human- and socio-technical context of testing through the lens of those interwoven elements: test suite and tools as technical infrastructure and collaborative factors and motivation as mindset. Drawing on empirical observations of previous work, this survey examines how these factors relate to each other. We want to understand which combination of factors can help developers strive and make the most of their ambitions to leverage the potential that software testing practices have. In this report, we construct a survey instrument to measure the factors that constitute the socio-technical context of testing experience. In addition, we state our hypotheses about how these factors impact testing experience and explain the considerations and process that led to the construction of the survey questions.

Problem

• Measure socio-technical factors in testing experience
• Explore how tools, tests, and mindsets shape testing
• Understand optimal factor combinations for effective testing

Innovation

• Survey instrument measures socio-technical testing factors
• Examines interplay of tools, tests, and developer mindsets
• Hypothesizes impact of factors on testing experience