🤖 AI Summary
To address the weak generalization capability and high computational cost of Restricted Kernel Machines (RKMs) on imbalanced, high-dimensional, and large-scale data, this paper proposes the Twin-Restricted Kernel Machine (TRKM). Methodologically, TRKM leverages the Fenchel–Young inequality to construct a conjugate feature duality, reformulating the optimization into a bivariate energy minimization framework that jointly optimizes visible and hidden variables. It integrates the Restricted Boltzmann Machine (RBM) energy function, kernel methods, regularized least squares, and Fenchel duality theory. Its key innovations include the first introduction of a conjugate feature duality mechanism and a twin-model architecture, substantially enhancing model robustness and scalability. Extensive experiments on UCI and KEEL benchmark datasets demonstrate consistent superiority over state-of-the-art methods. Moreover, TRKM achieves high accuracy and strong generalization in brain-age prediction—a challenging real-world neuroimaging task—validating its practical efficacy.
📝 Abstract
Restricted kernel machines (RKMs) have considerably improved generalization in machine learning. Recent advancements explored various techniques within the RKM framework, integrating kernel functions with least squares support vector machines (LSSVM) to mirror the energy function of restricted Boltzmann machines (RBM), leading to enhanced performance. However, RKMs may face challenges in generalization when dealing with unevenly distributed or complexly clustered data. Additionally, as the dataset size increases, the computational burden of managing high-dimensional feature spaces can become substantial, potentially hindering performance in large-scale datasets. To address these challenges, we propose twin restricted kernel machine (TRKM). TRKM combines the benefits of twin models with the robustness of the RKM framework to enhance classification and regression tasks. By leveraging the Fenchel-Young inequality, we introduce a novel conjugate feature duality, allowing the formulation of classification and regression problems in terms of dual variables. This duality provides an upper bound to the objective function of the TRKM problem, resulting in a new methodology under the RKM framework. The model uses an energy function similar to that of RBM, incorporating both visible and hidden variables corresponding to both classes. Additionally, the kernel trick is employed to map data into a high-dimensional feature space, where the model identifies an optimal separating hyperplane using a regularized least squares approach. Experiments on UCI and KEEL datasets confirm TRKM's superiority over baselines, showcasing its robustness and efficiency in handling complex data. Furthermore, We implemented the TRKM model on the brain age dataset, demonstrating its efficacy in predicting brain age.