Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
Cordial learning addresses distributed training with correlated agent data by enabling agents to share low-dimensional outputs to inform local model training, overcoming limitations of existing methods that ignore data structure or are infeasible due to communication/privacy constraints. This approach frames distributed learning as a game where agents' losses depend on peer models, and under linear assumptions, we prove convergence to a globally optimal solution despite a non-convex objective. Empirical results on structured multi-digit MNIST tasks validate cordial learning's effectiveness even in highly nonlinear scenarios.
Agents can achieve globally optimal solutions in distributed learning with correlated data by learning a game where they share low-dimensional signals, even with non-convex objectives.
We consider a distributed learning task with agents that have correlated data. Specifically, the label of an agent depends on the input of other agents for the same sample, and these inputs are also correlated. Correlated data is the reality when agents share the same environment. Existing decentralized methods, such as federated learning, ignore the structure of the problem and perform poorly on correlated data. On the other hand, centralized approaches are infeasible due to privacy and communication constraints. We introduce cordial (correlated and distributed) learning to address this gap by sharing only low-dimensional outputs between the agents while training local models to extract informative signals from peers. This distributed learning induces a game in which the loss function of each agent depends on the models of others. Assuming a linear model, we prove that cordial learning converges with probability one to a globally optimal solution, despite the nonconvex global objective. Experiments on structured multi-digit MNIST tasks demonstrate that cordial learning remains highly effective even in highly nonlinear settings.