Search papers, labs, and topics across Lattice.
The paper addresses the problem of uncertainty quantification (UQ) in federated learning (FL) under dual heterogeneity (both data and model heterogeneity) by proposing FedWQ-CP, a federated weighted quantile conformal prediction method. FedWQ-CP calibrates uncertainty at both the agent and server levels in a single communication round, balancing coverage performance and efficiency. Experiments on seven datasets demonstrate that FedWQ-CP achieves agent-wise and global coverage while generating smaller prediction sets/intervals compared to existing methods.
Achieve reliable uncertainty estimates in federated learning with FedWQ-CP, a surprisingly simple one-shot calibration method that handles both data and model heterogeneity.
Federated learning (FL) faces challenges in uncertainty quantification (UQ). Without reliable UQ, FL systems risk deploying overconfident models at under-resourced agents, leading to silent local failures despite seemingly satisfactory global performance. Existing federated UQ approaches often address data heterogeneity or model heterogeneity in isolation, overlooking their joint effect on coverage reliability across agents. Conformal prediction is a widely used distribution-free UQ framework, yet its applications in heterogeneous FL settings remains underexplored. We provide FedWQ-CP, a simple yet effective approach that balances empirical coverage performance with efficiency at both global and agent levels under the dual heterogeneity. FedWQ-CP performs agent-server calibration in a single communication round. On each agent, conformity scores are computed on calibration data and a local quantile threshold is derived. Each agent then transmits only its quantile threshold and calibration sample size to the server. The server simply aggregates these thresholds through a weighted average to produce a global threshold. Experimental results on seven public datasets for both classification and regression demonstrate that FedWQ-CP empirically maintains agent-wise and global coverage while producing the smallest prediction sets or intervals.
