Search papers, labs, and topics across Lattice.
This paper introduces a novel boosting algorithm that leverages list-decodable codes to enhance the efficiency of strong learning in concept classes satisfying a specific closure property. By utilizing a weak learner with a reduced number of calls, the algorithm achieves strong learning with only \(O(\log \frac{1}{\epsilon})\) calls to a \(γ\)-advantage weak learner, significantly improving upon the traditional \(O(\log(\frac{1}{\epsilon})/\gamma^2)\) complexity. The key result demonstrates that this approach not only circumvents established lower bounds but also provides a practical method for achieving strong learning with fewer resources in certain contexts.
Strong learning can be achieved with significantly fewer calls to weak learners by exploiting the structure of list-decodable codes.
Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989). Existing boosting algorithms construct a strong learner using $O(\log(\frac{1}ε)/γ^2)$ calls to a $γ$-advantage weak learner, and this round complexity is known to be optimal for generic boosters that succeed on all concept classes (Freund 1995). We show that this lower bound can be circumvented for concept classes that satisfy a mild closure property. Specifically, we present a new boosting algorithm that, for any class $\mathcal{F}$ closed under $O(\log \frac{1}γ)$-XOR, strong learns $\mathcal{F}$ using $O(\log \frac{1}ε)$ calls to a $γ$-advantage weak learner and a single batch of $\tilde{O}(\log(\frac{1}ε)/γ^2)$ additional samples. Our algorithm arises from a new and simple connection between boosting and list-decodable codes. Viewing the target function as a message, we run the weak learner on its encoding and view the resulting weak hypothesis as a corrupted codeword. Feeding this corrupted codeword to a list decoder, we obtain a small list of candidate hypotheses, at least one of which is a strong hypothesis for the original function. Using additional samples, we identify and output this strong hypothesis.
Stanford HAI