Search papers, labs, and topics across Lattice.
The paper introduces Federated Robust Curvature Optimization (FedRCO), a second-order federated learning optimizer designed to improve convergence speed and stability under non-IID data. FedRCO integrates a gradient anomaly monitor, a fail-safe resilience protocol, and a curvature-preserving adaptive aggregation strategy to mitigate exploding gradients and numerical instability. Experiments demonstrate that FedRCO achieves higher accuracy and faster convergence compared to existing first- and second-order methods in diverse non-IID scenarios.
Second-order federated learning can be made robust and practical: FedRCO overcomes instability issues and outperforms first-order methods in non-IID settings.
In this paper, we present Federated Robust Curvature Optimization (FedRCO), a novel second-order optimization framework designed to improve convergence speed and reduce communication cost in Federated Learning systems under statistical heterogeneity. Existing second-order optimization methods are often computationally expensive and numerically unstable in distributed settings. In contrast, FedRCO addresses these challenges by integrating an efficient approximate curvature optimizer with a provable stability mechanism. Specifically, FedRCO incorporates three key components: (1) a Gradient Anomaly Monitor that detects and mitigates exploding gradients in real-time, (2) a Fail-Safe Resilience protocol that resets optimization states upon numerical instability, and (3) a Curvature-Preserving Adaptive Aggregation strategy that safely integrates global knowledge without erasing the local curvature geometry. Theoretical analysis shows that FedRCO can effectively mitigate instability and prevent unbounded updates while preserving optimization efficiency. Extensive experiments show that FedRCO achieves superior robustness against diverse non-IID scenarios while achieving higher accuracy and faster convergence than both state-of-the-art first-order and second-order methods.
