Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces FedGRPO, a federated learning framework for optimizing foundation models by leveraging data from domain clients while preserving privacy. It frames the problem as a reinforcement learning task where a server model learns from scalar reward signals provided by expert clients selected using a competence-based confidence graph. FedGRPO aggregates these rewards using a federated group-relative loss function, achieving improved downstream accuracy and communication efficiency compared to existing federated foundation model approaches.
Forget sending model updates – FedGRPO trains foundation models by having clients simply score the *reasoning* of server-side models, slashing communication costs and boosting privacy.
One important direction of Federated Foundation Models (FedFMs) is leveraging data from small client models to enhance the performance of a large server-side foundation model. Existing methods based on model level or representation level knowledge transfer either require expensive local training or incur high communication costs and introduce unavoidable privacy risks. We reformulate this problem as a reinforcement learning style evaluation process and propose FedGRPO, a privacy preserving framework comprising two modules. The first module performs competence-based expert selection by building a lightweight confidence graph from auxiliary data to identify the most suitable clients for each question. The second module leverages the"Group Relative"concept from the Group Relative Policy Optimization (GRPO) framework by packaging each question together with its solution rationale into candidate policies, dispatching these policies to a selected subset of expert clients, and aggregating solely the resulting scalar reward signals via a federated group-relative loss function. By exchanging reward values instead of data or model updates, FedGRPO reduces privacy risk and communication overhead while enabling parallel evaluation across heterogeneous devices. Empirical results on diverse domain tasks demonstrate that FedGRPO achieves superior downstream accuracy and communication efficiency compared to conventional FedFMs baselines.