Privately Learning Decision Lists and a Differentially Private Winnow

2026-02-07

AI Summary

This work addresses the problem of efficiently learning decision lists and large-margin halfspace classifiers under differential privacy constraints. In the PAC learning model, the authors design a private algorithm that learns decision lists with sample complexity nearly matching that of non-private learners—the first such result in the PAC framework. In the online learning setting, they propose a differentially private variant of the Winnow algorithm that achieves a mistake bound polynomial in the logarithm of the dimension and the inverse of the margin for private halfspace learning. Furthermore, this approach is successfully extended to enable private online learning of decision lists, providing theoretical guarantees in both learning models while preserving privacy.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We give new differentially private algorithms for the classic problems of learning decision lists and large-margin halfspaces in the PAC and online models. In the PAC model, we give a computationally efficient algorithm for learning decision lists with minimal sample overhead over the best non-private algorithms. In the online model, we give a private analog of the influential Winnow algorithm for learning halfspaces with mistake bound polylogarithmic in the dimension and inverse polynomial in the margin. As an application, we describe how to privately learn decision lists in the online model, qualitatively matching state-of-the art non-private guarantees.

Problem

• differentially private
• decision lists
• halfspaces
• PAC learning
• online learning

Innovation

• differentially private learning
• decision lists
• online learning
• Winnow algorithm
• PAC learning