Neural Network Conversion of Machine Learning Pipelines
This work proposes a systematic method to convert non-neural machine learning pipelines—such as those based on random forests—into neural networks, enabling unified inference and joint optimization. Leveraging knowledge distillation, the approach treats the traditional model as a “teacher” that guides the training of a neural “student” network. The framework further integrates neural architecture search with a random forest–inspired hyperparameter selection strategy to optimize the student model. Notably, this is the first effort to employ an entire non-neural machine learning pipeline as the teacher in knowledge distillation, thereby extending the scope of this technique. Experimental evaluation across 100 OpenML tasks demonstrates that the student networks consistently replicate the performance of their teacher models, confirming the feasibility and effectiveness of the proposed conversion framework.