Why is Regularization Underused? An Empirical Study on Trust and Adoption of Statistical Methods

2026-04-03

AI Summary

Although regularization methods offer theoretical advantages in statistics, they remain underutilized in practice. This study investigates the barriers to trust and adoption of regularization techniques among data analysts, introducing the Technology Acceptance Model systematically into research on statistical method adoption for the first time. Drawing on an empirical analysis of 606 practitioners through a combination of survey data, a randomized controlled experiment, and structural equation modeling, the findings reveal that expert-written recommendations have limited impact on enhancing trust or intention to use these methods. Instead, actual adoption is primarily driven by perceived ease of implementation, perceived usefulness, and community norms. The research underscores the critical role of social and cognitive factors in methodological uptake and offers a novel perspective for facilitating the practical implementation of advanced statistical techniques.

Abstract

Statistical practice does not automatically follow methodological innovation. Regularization methods, widely advocated to reduce overfitting and stabilize inference, are readily available in modern software, but are not consistently used by data analysts. We investigate this implementation gap in a large-scale empirical study of trust in, and acceptance of, regularization techniques, based on $N = 606$ data analysts. Drawing on measurement frameworks from technology acceptance research, we survey practitioners and embed a randomized experiment to test whether written recommendation of regularization methods increases trust or intended use. We find no evidence of such an effect. Instead, adoption intentions are strongly associated with analysts' perceptions of ease of implementation and practical benefit, such as improved bias control or interpretability. Perceived social norms also emerge as a central driver. These results indicate that uptake of statistical methodology depends less on formal recommendations than on usability, perceived utility, and community practice.

Problem

• regularization
• trust
• adoption
• statistical methods
• implementation gap

Innovation

• regularization
• technology acceptance
• empirical study
• statistical practice
• implementation gap