🤖 AI Summary
This work addresses the limitations of traditional gradient boosting methods—such as limited expressiveness, sensitivity to hyperparameters, and difficulty in continual learning—when handling both structured and unstructured data. The authors propose Multiplicative Additive Neural Networks (MANN), which replace decision trees with shallow neural networks as base learners to enable unified modeling of images, audio, and tabular data. Notably, MANN incorporates capsule networks for the first time into feature extraction for structured data. Embedded within an enhanced gradient boosting framework, MANN integrates a continual learning mechanism and regularization strategies, substantially reducing sensitivity to learning rate and iteration count. Experimental results demonstrate that MANN outperforms strong baselines such as XGBoost across multiple benchmark datasets, exhibiting superior generalization and multimodal adaptability.
📝 Abstract
This paper extends and explains the Multiple Additive Neural Networks (MANN) methodology, an enhancement to the traditional Gradient Boosting framework, utilizing nearly shallow neural networks instead of decision trees as base learners. This innovative approach leverages neural network architectures, notably Convolutional Neural Networks (CNNs) and Capsule Neural Networks, to extend its application to both structured data and unstructured data such as images and audio. For structured data the advantages of capsule neural networks as feature extractors are used and combined with MANN as a classifier. MANN's unique architecture promotes continuous learning and integrates advanced heuristics to combat overfitting, ensuring robustness and reducing sensitivity to hyperparameter settings like learning rate and iterations. Our empirical studies reveal that MANN surpasses traditional methods such as Extreme Gradient Boosting (XGB) in accuracy across well-known datasets. This research demonstrates MANN's superior precision and generalizability, making it a versatile tool for diverse data types and complex learning environments.