TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning

2026-07-06

AI Summary

This work proposes TabPack, a novel MLP ensemble framework for tabular data that achieves strong out-of-the-box performance without requiring extensive hyperparameter tuning. Unlike conventional deep learning approaches for tabular data—which rely on laborious hyperparameter optimization—and existing MLP ensembles that use fixed hyperparameters and thus suffer from suboptimal performance and efficiency, TabPack trains multiple MLPs with diverse hyperparameters in parallel within a single run and dynamically selects the best ensemble members during training. Experimental results demonstrate that, on medium- to large-scale public datasets, TabPack with its default configuration matches or exceeds the performance of heavily tuned baseline methods, while its runtime on a standard laptop is substantially shorter than the total tuning time of baselines even when executed on high-end GPUs, significantly reducing both hyperparameter tuning costs and computational overhead.

Abstract

In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures. Existing methods of this kind use the same hyperparameters for all underlying MLPs, which requires hyperparameter tuning for achieving the best performance. In this work, we introduce TabPack, an efficient MLP ensemble with strong out-of-the-box performance and reduced reliance on traditional tuning. In a single run, TabPack samples and trains many MLPs with different hyperparameters efficiently in parallel and selects ensemble members on the fly during training. Thus, TabPack only requires specifying ranges from which to sample MLP hyperparameter rather than exact hyperparameter values, which naturally demands less precision for good performance. In experiments on medium-to-large public datasets, TabPack with default settings performs on par with extensively tuned prior methods, thus substantially reducing effort and compute resources needed to achieve competitive results on tabular tasks. Notably, running the default TabPack configuration on a modern MacBook took less time than tuning some baselines on an industry-grade GPU.

Problem

• tabular deep learning
• hyperparameter tuning
• efficient ensembles
• MLP
• out-of-the-box performance

Innovation

• hyperparameter ensembles
• tabular deep learning
• efficient MLP ensemble
• out-of-the-box performance
• parallel hyperparameter sampling