Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Partition-Induced Covariate-shift Correction (PIcsC), a regularization framework that addresses covariate shift across training-data partitions by utilizing the Fisher Information Matrix (FIM) to approximate partition divergence. This approach is applicable to both centralized and decentralized learning environments, requiring only partition-local gradient statistics, thus enhancing model performance without accessing raw data. Experimental results show that PIcsC significantly reduces performance degradation due to fragmentation and outperforms existing federated learning methods by 3-5 percentage points across multiple benchmarks.
PIcsC reduces fragmentation-induced performance degradation by over 20% while outperforming leading federated learning methods without requiring client-specific personalization.
Covariate shift across training-data partitions biases model selection and parameter estimation in cross-validation, lifelong learning, and federated learning. We propose \textit{Partition-Induced Covariate-shift Correction} (\texttt{PIcsC}), a Fisher information-based regularization framework that mitigates distribution mismatch between data partitions and a reference distribution. \texttt{PIcsC} approximates partition divergence using the Fisher Information Matrix (FIM) and incorporates the resulting statistic as a regularizer during optimization. The same formulation applies to both centrally partitioned datasets (batches or cross-validation folds) and inherently distributed data (federated clients or decentralized nodes), requiring only partition-local gradient statistics rather than raw data. We further introduce a conditional adaptation mechanism that combines FIM shift with KL divergence to detect significant distribution shifts and activates regularization only when necessary. Experiments on more than 40 datasets demonstrate consistent improvements under both natural and synthetic covariate shift. On fragmented batch and fold settings, \texttt{PIcsC} reduces fragmentation-induced performance degradation by more than 20\% and 25\%, respectively. On seven federated learning benchmarks, it consistently outperforms FedAvg, FedProx, and SCAFFOLD by 3 -5 percentage points without requiring client-specific personalization. These results demonstrate that Fisher information provides an effective and unified mechanism for mitigating partition-induced covariate shift across both centralized and distributed learning.