TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data

2026-07-10

AI Summary

This work addresses the high computational cost and deployment inefficiency of deep ensemble models on large-scale tabular data. The authors propose TabLoRA, the first approach to integrate low-rank adaptation (LoRA) into neural ensemble learning for tabular data. By sharing a common backbone network and equipping each predictor with a lightweight, trainable low-rank adapter module, TabLoRA preserves ensemble diversity without replicating backbone parameters. This design substantially reduces both parameter count and computational overhead. Empirical results demonstrate that TabLoRA outperforms state-of-the-art gradient-boosted decision tree (GBDT) methods and existing deep learning approaches across multiple benchmark datasets, achieving superior predictive performance while maintaining high efficiency under identical resource constraints.

Abstract

Tabular learning is still dominated by gradient-boosted decision trees (GBDTs), while recent deep learning approaches have become increasingly competitive. However, applying deep tabular models to large-scale datasets remains challenging, as large sample sizes, high feature dimensionality, or many target classes can introduce substantial computational cost. We propose TabLoRA, a parameter-efficient trainable neural ensemble for large-scale tabular learning. Instead of using fully independent ensemble backbones, TabLoRA shares a common backbone across predictors and introduces predictor-specific low-rank adaptations, enabling ensemble-style prediction without full parameter duplication. Across benchmarks, TabLoRA achieves a favorable balance between predictive performance and practical efficiency compared with GBDT methods and recent deep learning baselines under the same resource constraints. Memory analysis and ablation studies further show that the proposed design improves the feasibility of neural ensemble learning while preserving much of the benefit of full ensembles.

Problem

• tabular learning
• large-scale datasets
• computational cost
• parameter efficiency
• neural ensemble

Innovation

• parameter-efficient
• low-rank adaptation
• neural ensemble
• tabular learning
• shared backbone