Beyond KAN: Introducing KarSein for Adaptive High-Order Feature Interaction Modeling in CTR Prediction

2024-08-16

arXiv.org

AI Summary

Existing CTR prediction models struggle with modeling high-order explicit feature interactions due to reliance on pre-specified interaction orders, poor interpretability, and high computational overhead. Method: We propose KarSein—a novel model integrating symbolic regression-guided Kolmogorov–Arnold networks—enabling adaptive, sparse, multiplicative high-order interaction learning directly from embedding inputs without requiring prior order specification. Its structurally sparse parameterization and embedding-compatible architecture ensure global interpretability while drastically improving efficiency. Contributions/Results: Extensive experiments on multiple CTR benchmark datasets demonstrate that KarSein achieves significant gains in prediction accuracy, reduces computational cost by 30%–50%, supports automatic pruning of redundant features, and enables real-time inference—all while preserving end-to-end interpretability of learned interactions.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Modeling feature interactions is crucial for click-through rate (CTR) prediction, particularly when it comes to high-order explicit interactions. Traditional methods struggle with this task because they often predefine a maximum interaction order, which relies heavily on prior knowledge and can limit the model's effectiveness. Additionally, modeling high-order interactions typically leads to increased computational costs. Therefore, the challenge lies in adaptively modeling high-order feature interactions while maintaining efficiency. To address this issue, we introduce Kolmogorov-Arnold Represented Sparse Efficient Interaction Network (KarSein), designed to optimize both predictive accuracy and computational efficiency. We firstly identify limitations of directly applying Kolmogorov-Arnold Networks (KAN) to CTR and then introduce KarSein to overcome these issues. It features a novel architecture that reduces the computational costs of KAN and supports embedding vectors as feature inputs. Additionally, KarSein employs guided symbolic regression to address the challenge of KAN in spontaneously learning multiplicative relationships. Extensive experiments demonstrate KarSein's superior performance, achieving significant predictive accuracy with minimal computational overhead. Furthermore, KarSein maintains strong global explainability while enabling the removal of redundant features, resulting in a sparse network structure. These advantages also position KarSein as a promising method for efficient inference.

Problem

• Click-Through Rate Prediction
• Complex Feature Relationships
• Resource-Efficient Methods

Innovation

• KarSein
• Computational Efficiency
• Predictive Accuracy