Search papers, labs, and topics across Lattice.
This paper characterizes learnability under distributional adversaries, where data-generating distributions are adaptively chosen from a fixed family U, by establishing that learnability is equivalent to a notion called generalized smoothness. They show that a distribution family admits VC-dimension-dependent regret bounds for every finite-VC hypothesis class if and only if it is generalized smooth, and provide universal algorithms achieving low regret under any generalized smooth adversary. Furthermore, the paper demonstrates that generalized smoothness also characterizes private learnability under distributional constraints, linking online learning and differential privacy.
Learnability under distributional adversaries hinges on a single property – generalized smoothness – which also dictates private learnability, unifying online learning and differential privacy under a common framework.
Understanding minimal assumptions that enable learning and generalization is perhaps the central question of learning theory. Several celebrated results in statistical learning theory, such as the VC theorem and Littlestone's characterization of online learnability, establish conditions on the hypothesis class that allow for learning under independent data and adversarial data, respectively. Building upon recent work bridging these extremes, we study sequential decision making under distributional adversaries that can adaptively choose data-generating distributions from a fixed family $U$ and ask when such problems are learnable with sample complexity that behaves like the favorable independent case. We provide a near complete characterization of families $U$ that admit learnability in terms of a notion known as generalized smoothness i.e. a distribution family admits VC-dimension-dependent regret bounds for every finite-VC hypothesis class if and only if it is generalized smooth. Further, we give universal algorithms that achieve low regret under any generalized smooth adversary without explicit knowledge of $U$. Finally, when $U$ is known, we provide refined bounds in terms of a combinatorial parameter, the fragmentation number, that captures how many disjoint regions can carry nontrivial mass under $U$. These results provide a nearly complete understanding of learnability under distributional adversaries. In addition, building upon the surprising connection between online learning and differential privacy, we show that the generalized smoothness also characterizes private learnability under distributional constraints.
