$L_1$-norm Regularized Indefinite Kernel Logistic Regression

2025-10-29

AI Summary

To address the challenge of simultaneously achieving sparsity, interpretability, and generalization in indefinite kernel logistic regression (IKLR), this paper introduces the $L_1$-norm regularization into the IKLR framework for the first time, yielding the Sparse Indefinite Kernel Logistic Regression (S-IKLR) model. To tackle the resulting nonsmooth and nonconvex optimization problem, we propose an efficient proximal linearization-based algorithm with theoretical convergence guarantees. S-IKLR leverages the expressive power of indefinite kernels to capture complex data structures while enforcing sparsity via $L_1$ regularization, thereby substantially reducing the number of nonzero parameters. Extensive experiments on multiple benchmark datasets demonstrate that S-IKLR achieves superior classification accuracy compared to state-of-the-art IKLR and sparse kernel methods. Moreover, it attains 30–60% higher model sparsity, leading to significantly improved interpretability and generalization performance.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Non-convex Optimization
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Kernel logistic regression (KLR) is a powerful classification method widely applied across diverse domains. In many real-world scenarios, indefinite kernels capture more domain-specific structural information than positive definite kernels. This paper proposes a novel $L_1$-norm regularized indefinite kernel logistic regression (RIKLR) model, which extends the existing IKLR framework by introducing sparsity via an $L_1$-norm penalty. The introduction of this regularization enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape. To address these challenges, a theoretically grounded and computationally efficient proximal linearized algorithm is developed. Experimental results on multiple benchmark datasets demonstrate the superior performance of the proposed method in terms of both accuracy and sparsity.

Problem

• Develop L1-regularized indefinite kernel logistic regression model
• Address nonsmooth nonconvex optimization in kernel methods
• Enhance classification accuracy and feature sparsity simultaneously

Innovation

• L1-norm regularized indefinite kernel logistic regression model
• Proximal linearized algorithm for nonsmooth nonconvex optimization
• Sparsity-enhanced interpretability and generalization performance