Search papers, labs, and topics across Lattice.
This paper presents quasipolynomial-time algorithms for learning $\mathsf{AC}^0$ circuits when the input distribution comes from any graphical model with polynomial growth exhibiting strong spatial mixing, moving beyond the traditional i.i.d. assumption. The key innovation lies in circumventing Fourier analysis, a standard tool for learning under uniform distributions, by developing new sampling algorithms that transfer low-degree polynomial approximation results from the uniform setting to graphical models. This approach extends to other function classes like monotone functions and halfspaces, broadening the applicability of learning algorithms under correlated data.
Learning constant-depth circuits is possible even with correlated data from graphical models, challenging the reliance on i.i.d. assumptions in learning theory.
In a landmark result, Linial, Mansour and Nisan (J. ACM 1993) gave a quasipolynomial-time algorithm for learning constant-depth circuits given labeled i.i.d. samples under the uniform distribution. Their work has had a deep and lasting legacy in computational learning theory, in particular introducing the $\textit{low-degree algorithm}$. However, an important critique of many results and techniques in the area is the reliance on product structure, which is unlikely to hold in realistic settings. Obtaining similar learning guarantees for more natural correlated distributions has been a longstanding challenge in the field. In particular, we give quasipolynomial-time algorithms for learning $\mathsf{AC}^0$ substantially beyond the product setting, when the inputs come from any graphical model with polynomial growth that exhibits strong spatial mixing. The main technical challenge is in giving a workaround to Fourier analysis, which we do by showing how new sampling algorithms allow us to transfer statements about low-degree polynomial approximation under the uniform setting to graphical models. Our approach is general enough to extend to other well-studied function classes, like monotone functions and halfspaces.
