Twin Restricted Kernel Machines for Multiview Classification

2025-12-11

AI Summary

To address key bottlenecks in multi-view SVMs—including difficulty modeling decision boundaries in high-dimensional spaces, high computational cost, and poor robustness to view inconsistency—this paper proposes Twin Multi-View Regularized Kernel Machines (TMvRKM). TMvRKM innovatively integrates early- and late-fusion strategies within a regularized least-squares optimization framework and introduces a cross-view error coupling term to explicitly enforce inter-view consistency. Unlike conventional kernel methods, TMvRKM bypasses complex dual optimization, substantially reducing training complexity. Its coupled loss function jointly optimizes discriminability and consistency. Extensive experiments on UCI, KEEL, and AwA benchmarks demonstrate that TMvRKM consistently outperforms state-of-the-art multi-view SVMs and deep baselines. Statistical analysis confirms significant improvements in generalization performance and training efficiency.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multi-view learning (MVL) is an emerging field in machine learning that focuses on improving generalization performance by leveraging complementary information from multiple perspectives or views. Various multi-view support vector machine (MvSVM) approaches have been developed, demonstrating significant success. Moreover, these models face challenges in effectively capturing decision boundaries in high-dimensional spaces using the kernel trick. They are also prone to errors and struggle with view inconsistencies, which are common in multi-view datasets. In this work, we introduce the multiview twin restricted kernel machine (TMvRKM), a novel model that integrates the strengths of kernel machines with the multiview framework, addressing key computational and generalization challenges associated with traditional kernel-based approaches. Unlike traditional methods that rely on solving large quadratic programming problems (QPPs), the proposed TMvRKM efficiently determines an optimal separating hyperplane through a regularized least squares approach, enhancing both computational efficiency and classification performance. The primal objective of TMvRKM includes a coupling term designed to balance errors across multiple views effectively. By integrating early and late fusion strategies, TMvRKM leverages the collective information from all views during training while remaining flexible to variations specific to individual views. The proposed TMvRKM model is rigorously tested on UCI, KEEL, and AwA benchmark datasets. Both experimental results and statistical analyses consistently highlight its exceptional generalization performance, outperforming baseline models in every scenario.

Problem

• Enhances generalization in multi-view learning with complementary data
• Addresses kernel trick challenges in high-dimensional decision boundaries
• Improves computational efficiency via regularized least squares over QPPs

Innovation

• Integrates kernel machines with multiview framework for classification
• Uses regularized least squares for efficient hyperplane determination
• Balances errors across views with early and late fusion strategies