🤖 AI Summary
This study addresses the challenge of enhancing generalization performance in binary classification tasks by proposing a novel convex loss function that integrates pattern correlation. We theoretically demonstrate that this formulation constitutes a generalized extension of standard losses. Methodologically, Particle Swarm Optimization is employed to solve the primal problem, with model performance evaluated via nested cross-validation. A key contribution lies in elucidating the mechanism through which pattern correlation influences generalization. Experimental results indicate that the proposed loss achieves generalization levels comparable to standard benchmarks on small-sample datasets. Consequently, this work provides robust theoretical support and establishes a new paradigm for optimizing classification models in data-scarce scenarios, effectively bridging the gap between correlation-aware learning and practical small-data applications.
📝 Abstract
We propose multiple new convex losses for SVM and Neural Networks, applied to binary classification tasks. While there are practical limitations in exploiting them with the dual SVM models, we are able to use them with SVM primal formulation and Neural Networks. In detail, the primal SVM problem with the modified losses has been solved with the Particle Swarm Optimization algorithm. We prove that the proposed losses are a generalization of the standard loss, and we experiment them with several small data-sets. This preliminary study shows that using pattern correlations
inside the loss function could in theory enhance the generalization performances on some data-sets. To evaluate the performance of each loss, we adopt a Nested Cross-Validation procedure. Results show that generalization measures are the same with or without the new losses.