Search papers, labs, and topics across Lattice.
This paper introduces the framework of simulatable processes, allowing learners to utilize a simulator that approximates the data-generating distribution, even when the underlying process is complex and dependent. The authors demonstrate that access to such simulators enables the recovery of classical learning guarantees, specifically error bounds linked to the VC dimension, which are typically reliant on independent data assumptions. Notably, they present an algorithm capable of learning any VC class under all processes that can be sampled in bounded polynomial time, with regret determined by the time-bounded Kolmogorov complexity of the process, thus expanding the classical PAC model's applicability.
Access to simulators can yield classical learning guarantees even in the presence of complex data dependencies, fundamentally shifting our understanding of learning theory.
Understanding the minimal assumptions necessary for generalization is the fundamental question in learning theory. Unfortunately, most results rely heavily on independence (or some proxy thereof) of the data-generating process, while results for strongly dependent data are far more limited. Towards addressing this gap, we introduce the framework of simulatable processes, where the learner has access to a simulator that approximates the distribution generating the data (which may be an arbitrarily complex and dependent process). Surprisingly, given access to such a simulator, we show that we can recover the same learning guarantees as in the classical setting with independent data, namely, error bounds that depend on the VC dimension. Further, we use this framework to study the power of conditional sampling and show strict statistical and computational advantages in this setting. As a highlight of our framework, we exhibit a single algorithm that simultaneously learns any given VC class under all processes samplable in bounded polynomial time, with regret controlled by the time-bounded Kolmogorov complexity of the process. This provides a significant conceptual broadening of the classical PAC model.
Microsoft Research