Towards Convexity in Anomaly Detection: A New Formulation of SSLM with Unique Optimal Solutions

2024-10-31

arXiv.org

AI Summary

Mainstream anomaly detection methods—such as Support Vector Data Description (SVDD) and Semi-Supervised Learning Machines (SSLM)—suffer from non-convex objective functions, hindering theoretical analysis of global optima and limiting scalability. Method: This paper proposes the first strictly convex reformulation of SSLM. By constructing an equivalent convex quadratic programming (QP) formulation, we achieve the first rigorous convexification of SSLM. Contributions: We establish necessary and sufficient conditions for the existence and uniqueness of the optimal solution; rigorously characterize the ν-property—i.e., the precise control exerted by the hyperparameter ν over support vector proportion and error distribution; and systematically delineate pathological cases and trivial-solution boundaries. The resulting convex QP is efficiently solvable via standard optimization routines, thereby providing a new foundation for large-scale anomaly detection that simultaneously ensures theoretical interpretability, algorithmic robustness, and computational scalability.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Non-convex Optimization
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

An unsolved issue in widely used methods such as Support Vector Data Description (SVDD) and Small Sphere and Large Margin SVM (SSLM) for anomaly detection is their nonconvexity, which hampers the analysis of optimal solutions in a manner similar to SVMs and limits their applicability in large-scale scenarios. In this paper, we introduce a novel convex SSLM formulation which has been demonstrated to revert to a convex quadratic programming problem for hyperparameter values of interest. Leveraging the convexity of our method, we derive numerous results that are unattainable with traditional nonconvex approaches. We conduct a thorough analysis of how hyperparameters influence the optimal solution, pointing out scenarios where optimal solutions can be trivially found and identifying instances of ill-posedness. Most notably, we establish connections between our method and traditional approaches, providing a clear determination of when the optimal solution is unique -- a task unachievable with traditional nonconvex methods. We also derive the { u}-property to elucidate the interactions between hyperparameters and the fractions of support vectors and margin errors in both positive and negative classes.

Problem

• Addressing nonconvexity issues in anomaly detection methods like SVDD and SSLM
• Developing a convex SSLM formulation with unique optimal solutions
• Analyzing hyperparameter effects and connections to traditional anomaly detection approaches

Innovation

• Convex SSLM formulation for anomaly detection
• Converts to convex quadratic programming problem
• Enables unique optimal solution analysis