Boosting with List-Decodable Codes

2026-07-06

AI Summary

This work addresses the round complexity of boosting algorithms for concept classes satisfying XOR closure. Classical boosting requires $O(\log(1/\varepsilon)/\gamma^2)$ calls to a weak learner to achieve error $\varepsilon$, matching known theoretical lower bounds. The authors establish, for the first time, a connection between boosting and list-decodable codes for such concept classes: by treating the target function as an encodable message and leveraging the weak learner’s outputs together with list decoding techniques, they efficiently identify a strong hypothesis from a candidate set. This approach breaks the classical round-complexity barrier, achieving the desired accuracy with only $O(\log(1/\varepsilon))$ rounds of weak learner queries, supplemented by $\tilde{O}(\log(1/\varepsilon)/\gamma^2)$ additional samples for verification, thereby significantly improving learning efficiency.

Abstract

Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989). Existing boosting algorithms construct a strong learner using $O(\log(\frac{1}ε)/γ^2)$ calls to a $γ$-advantage weak learner, and this round complexity is known to be optimal for generic boosters that succeed on all concept classes (Freund 1995). We show that this lower bound can be circumvented for concept classes that satisfy a mild closure property. Specifically, we present a new boosting algorithm that, for any class $\mathcal{F}$ closed under $O(\log \frac{1}γ)$-XOR, strong learns $\mathcal{F}$ using $O(\log \frac{1}ε)$ calls to a $γ$-advantage weak learner and a single batch of $\tilde{O}(\log(\frac{1}ε)/γ^2)$ additional samples. Our algorithm arises from a new and simple connection between boosting and list-decodable codes. Viewing the target function as a message, we run the weak learner on its encoding and view the resulting weak hypothesis as a corrupted codeword. Feeding this corrupted codeword to a list decoder, we obtain a small list of candidate hypotheses, at least one of which is a strong hypothesis for the original function. Using additional samples, we identify and output this strong hypothesis.

Problem

• boosting
• list-decodable codes
• weak learner
• concept class
• round complexity

Innovation

• boosting
• list-decodable codes
• weak learning
• closure property
• sample complexity