Improved Replicable Boosting with Majority-of-Majorities

2025-01-30

AI Summary

This paper addresses the prohibitively high sample complexity of Boosting algorithms in replicable learning. We propose a novel two-level majority-voting mechanism—termed “majority-of-majorities”—built upon Impagliazzo et al.’s (2022) replicable Boosting framework. Our method integrates weak learners via deterministic majority voting at both the base-learner and ensemble levels, preserving strong replicability while drastically reducing sample complexity from polynomial to significantly lower order. Theoretically, our design breaks the sample-efficiency bottleneck inherent in prior replicable Boosting approaches, yielding tighter generalization error bounds and stronger cross-platform result consistency guarantees. Empirical evaluation demonstrates substantial improvements in stability and reproducibility under limited-sample regimes.

Technology Category

• Machine Learning: Ensemble Methods
• Multiagent Systems: Mechanism Design
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We introduce a new replicable boosting algorithm which significantly improves the sample complexity compared to previous algorithms. The algorithm works by doing two layers of majority voting, using an improved version of the replicable boosting algorithm introduced by Impagliazzo et al. [2022] in the bottom layer.

Problem

• New Boosting Algorithm
• Sample Size Reduction
• Performance Improvement

Innovation

• Upgraded Boosting Algorithm
• Two-round Voting Strategy
• Enhanced Efficiency and Accuracy