Search papers, labs, and topics across Lattice.
This paper establishes a theoretical framework for unsupervised federated learning by modeling interactions among multiple students receiving distinct data realizations. The authors demonstrate that these interactions enhance learning performance, with noisier students needing fewer samples to identify underlying patterns, while less noisy students align more closely with the true signal. They derive optimal Bayesian conditions for teacher recovery based on sample complexity, noise levels, and interaction strength, supported by numerical simulations that map the dynamics to a Restricted Boltzmann Machine framework.
Interactions among federated learning students can significantly boost performance, enabling noisier participants to learn effectively with fewer samples.
We introduce a theoretical framework for analyzing federated learning in a generative setting through a teacher-multiple interacting students scenario, in which each student receives a distinct realization of the data, either through a different noise corruption or by accessing a different subset, possibly of varying size. Using theoretical tools in equilibrium disordered system, we analytically show that interactions among students systematically enhance learning performance: highly noisy students require fewer samples to recover the underlying pattern, while low-noise students achieve a larger overlap with the ground-truth signal. We derive the optimal Bayesian conditions for teacher recovery as functions of the sample complexity, noise level, and interaction strength, and validate these predictions through numerical simulations. The resulting dynamics can be mapped onto equilibrium sampling in a Restricted Boltzmann Machine with a structured hidden layer, providing a principled theoretical understanding of how interactions improve distributed generative modeling.
