A Reproducible Framework for Bias-Resistant Machine Learning on Small-Sample Neuroimaging Data.

2026-02-02

arXiv.org

AI Summary

This study addresses the optimistic bias and compromised reproducibility in few-shot neuroimaging analysis, where conventional cross-validation often conflates model selection and evaluation by sharing data splits. To mitigate this, the authors propose a bias-resistant, interpretable machine learning framework that integrates domain knowledge–driven feature engineering, importance-guided feature selection, rigorous nested cross-validation, and calibrated decision threshold optimization. Applied to high-dimensional structural MRI data for predicting cognitive outcomes following deep brain stimulation, the approach achieves a nested cross-validated balanced accuracy of 0.660 ± 0.068 using a compact and interpretable feature subset, thereby significantly enhancing the reliability and transparency of small-sample biomedical modeling.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Computer Vision: Bias, Fairness & Privacy
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Machine learning and data science for the Web

Abstract

We introduce a reproducible, bias-resistant machine learning framework that integrates domain-informed feature engineering, nested cross-validation, and calibrated decision-threshold optimization for small-sample neuroimaging data. Conventional cross-validation frameworks that reuse the same folds for both model selection and performance estimation yield optimistically biased results, limiting reproducibility and generalization. Demonstrated on a high-dimensional structural MRI dataset of deep brain stimulation cognitive outcomes, the framework achieved a nested-CV balanced accuracy of 0.660\,$\pm$\,0.068 using a compact, interpretable subset selected via importance-guided ranking. By combining interpretability and unbiased evaluation, this work provides a generalizable computational blueprint for reliable machine learning in data-limited biomedical domains.

Problem

• bias
• small-sample
• neuroimaging
• reproducibility
• machine learning

Innovation

• bias-resistant machine learning
• nested cross-validation
• small-sample neuroimaging
• interpretable feature selection
• calibrated decision-threshold optimization