Elastic-Net Multiple Kernel Learning: Combining Multiple Data Sources for Prediction

2025-12-12

AI Summary

To address predictive modeling with multi-source heterogeneous data (e.g., multimodal neuroimaging), this paper proposes an elastic-net-regularized multiple kernel learning (MKL) framework that jointly optimizes kernel weights and prediction parameters. Methodologically, it introduces the first analytical kernel weight update algorithm, circumventing the conventional two-stage paradigm. By employing combined L₁ + L₂ regularization, the approach simultaneously enforces sparsity in kernel weights and promotes collaborative selection of relevant kernels, thereby substantially enhancing model interpretability. In three neuroimaging prediction tasks, the method consistently outperforms L₁-MKL and achieves performance comparable to or better than single-kernel SVMs or kernel ridge regression (KRR), while yielding sparser and more biologically meaningful kernel weight configurations. The implementation is publicly available within the PRoNTo toolbox.

Technology Category

• Machine Learning: Kernel Methods
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Multiple Kernel Learning (MKL) models combine several kernels in supervised and unsupervised settings to integrate multiple data representations or sources, each represented by a different kernel. MKL seeks an optimal linear combination of base kernels that maximizes a generalized performance measure under a regularization constraint. Various norms have been used to regularize the kernel weights, including $l1$, $l2$ and $lp$, as well as the "elastic-net" penalty, which combines $l1$- and $l2$-norm to promote both sparsity and the selection of correlated kernels. This property makes elastic-net regularized MKL (ENMKL) especially valuable when model interpretability is critical and kernels capture correlated information, such as in neuroimaging. Previous ENMKL methods have followed a two-stage procedure: fix kernel weights, train a support vector machine (SVM) with the weighted kernel, and then update the weights via gradient descent, cutting-plane methods, or surrogate functions. Here, we introduce an alternative ENMKL formulation that yields a simple analytical update for the kernel weights. We derive explicit algorithms for both SVM and kernel ridge regression (KRR) under this framework, and implement them in the open-source Pattern Recognition for Neuroimaging Toolbox (PRoNTo). We evaluate these ENMKL algorithms against $l1$-norm MKL and against SVM (or KRR) trained on the unweighted sum of kernels across three neuroimaging applications. Our results show that ENMKL matches or outperforms $l1$-norm MKL in all tasks and only underperforms standard SVM in one scenario. Crucially, ENMKL produces sparser, more interpretable models by selectively weighting correlated kernels.

Problem

• Develops Elastic-Net MKL for combining multiple data sources
• Improves model interpretability by weighting correlated kernels sparsely
• Provides analytical weight updates for SVM and kernel ridge regression

Innovation

• Elastic-net penalty combines l1 and l2 norms for sparsity and correlation
• Analytical update simplifies kernel weight optimization in MKL
• Implemented in PRoNTo toolbox for SVM and kernel ridge regression