Best Practices for Machine Learning Experimentation in Scientific Applications

📅 2025-11-26
📈 Citations: 0
Influential: 0
📄 PDF

career value

194K/year
🤖 AI Summary
Scientific machine learning experiments often suffer from distorted performance evaluations due to poor experimental design and inconsistent documentation. To address this, we propose a principled framework for ML experimentation tailored to scientific research, encompassing data preprocessing, model selection, cross-validation, and reporting—emphasizing reproducibility, fair comparison, and transparency. Our key contributions include two novel quantitative metrics: the Logarithmic Overfitting Ratio (LOR) and Composite Overfitting Score (COS), which jointly characterize overfitting severity and instability across cross-validation folds. Complementing these, we introduce standardized preprocessing protocols, rigorously defined strong baselines, and modular visualization templates for diagnostic analysis. Empirical evaluation demonstrates that our framework substantially enhances experimental rigor, reproducibility, and result credibility in scientific ML. It further enables robust performance assessment and cross-study comparability, providing systematic support for establishing reliable benchmarks.

Technology Category

Application Category

📝 Abstract
Machine learning (ML) is increasingly adopted in scientific research, yet the quality and reliability of results often depend on how experiments are designed and documented. Poor baselines, inconsistent preprocessing, or insufficient validation can lead to misleading conclusions about model performance. This paper presents a practical and structured guide for conducting ML experiments in scientific applications, focussing on reproducibility, fair comparison, and transparent reporting. We outline a step-by-step workflow, from dataset preparation to model selection and evaluation, and propose metrics that account for overfitting and instability across validation folds, including the Logarithmic Overfitting Ratio (LOR) and the Composite Overfitting Score (COS). Through recommended practices and example reporting formats, this work aims to support researchers in establishing robust baselines and drawing valid evidence-based insights from ML models applied to scientific problems.
Problem

Research questions and friction points this paper is trying to address.

Ensuring reproducibility and fair comparison in scientific ML experiments
Addressing misleading conclusions from poor baselines and validation practices
Providing structured workflow for robust model evaluation in research
Innovation

Methods, ideas, or system contributions that make the work stand out.

Structured workflow for ML experiment reproducibility
Metrics like LOR and COS for overfitting assessment
Guidelines for robust baselines and transparent reporting
🔎 Similar Papers
2024-06-25arXiv.orgCitations: 0
U
Umberto Michelucci
Computer Science Department, Lucerne University of Applied Sciences and Arts, Luzern 6002, Switzerland
F
Francesca Venturini
Institute of Applied Mathematics and Physics, ZHAW - Zurich University of Applied Sciences, Winterthur 8400, Zurich, Switzerland