🤖 AI Summary
To address the challenges of implicit margin optimization and insufficient decision-boundary robustness in multiclass SVMs, this paper proposes a novel unified multiclass SVM framework. It explicitly incorporates the maximum-margin principle into the modeling process for the first time, jointly optimizing pairwise losses across all class pairs via structured optimization and introducing margin-driven regularization. Theoretically, the framework unifies margin maximization with multiclass loss minimization; practically, it can be seamlessly integrated into deep networks as a drop-in replacement or enhancement for the softmax layer. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method significantly outperforms one-vs-one (OvO), one-vs-rest (OvR), and state-of-the-art multiclass SVM approaches, achieving consistent improvements in both generalization performance and adversarial robustness.
📝 Abstract
Support Vector Machine (SVM) stands out as a prominent machine learning technique widely applied in practical pattern recognition tasks. It achieves binary classification by maximizing the "margin", which represents the minimum distance between instances and the decision boundary. Although many efforts have been dedicated to expanding SVM for multi-class case through strategies such as one versus one and one versus the rest, satisfactory solutions remain to be developed. In this paper, we propose a novel method for multi-class SVM that incorporates pairwise class loss considerations and maximizes the minimum margin. Adhering to this concept, we embrace a new formulation that imparts heightened flexibility to multi-class SVM.
Furthermore, the correlations between the proposed method and multiple forms of multi-class SVM are analyzed. The proposed regularizer, akin to the concept of "margin", can serve as a seamless enhancement over the softmax in deep learning, providing guidance for network parameter learning. Empirical evaluations demonstrate the effectiveness and superiority of our proposed
method over existing multi-classification methods. Complete version is available at https://arxiv.org/pdf/2312.06578.pdf. Code is available at https://github.com/zz-haooo/M3SVM.