Hybrid(Penalized Regression and MLP) Models for Outcome Prediction in HDLSS Health Data

2025-12-02

AI Summary

To address the low prediction accuracy and AUC in diabetes classification on high-dimensional, low-sample-size (HDLSS) NHANES health data, this paper proposes an XGBoost-MLP hybrid model. It first employs XGBoost for robust feature encoding and importance-based dimensionality reduction, yielding compact, dense low-dimensional representations; these are then fed into a lightweight multilayer perceptron (MLP) for nonlinear classification. This architecture synergistically integrates the interpretability and feature selection capability of tree-based models with the strong nonlinear modeling capacity of neural networks. Evaluated on a rigorously preprocessed NHANES subset, the model achieves an AUC of 0.892 and a balanced accuracy of 0.831—significantly outperforming logistic regression, random forest, and standalone XGBoost baselines (p < 0.01). The implementation code and full reproducibility scripts are publicly available.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Humans and AI: Human-in-the-loop Machine Learning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

I present an application of established machine learning techniques to NHANES health survey data for predicting diabetes status. I compare baseline models (logistic regression, random forest, XGBoost) with a hybrid approach that uses an XGBoost feature encoder and a lightweight multilayer perceptron (MLP) head. Experiments show the hybrid model attains improved AUC and balanced accuracy compared to baselines on the processed NHANES subset. I release code and reproducible scripts to encourage replication.

Problem

• Predict diabetes status using NHANES health survey data
• Compare hybrid XGBoost-MLP model with baseline machine learning methods
• Improve prediction performance measured by AUC and balanced accuracy

Innovation

• Hybrid model combines XGBoost encoder with MLP head
• Improves AUC and balanced accuracy over baseline models
• Applied to processed NHANES health survey data