Search papers, labs, and topics across Lattice.
The paper introduces Fractional-Order Federated Averaging (FOFedAvg), a FedAvg variant that uses Fractional-Order Stochastic Gradient Descent (FOSGD) to incorporate long-range relationships and historical information into the model updates. This approach aims to improve communication efficiency, accelerate convergence, and mitigate instability in federated learning scenarios with non-IID data. Empirical results on benchmark datasets demonstrate that FOFedAvg is competitive with or outperforms existing federated optimization algorithms, and the paper provides theoretical convergence guarantees for FOFedAvg under standard assumptions.
By remembering more of the past, fractional-order updates can significantly boost federated learning performance on heterogeneous data.
Federated learning (FL) allows remote clients to train a global model collaboratively while protecting client privacy. Despite its privacy-preserving benefits, FL has significant drawbacks, including slow convergence, high communication cost, and non-independent-and-identically-distributed (non-IID) data. In this work, we present a novel FedAvg variation called Fractional-Order Federated Averaging (FOFedAvg), which incorporates Fractional-Order Stochastic Gradient Descent (FOSGD) to capture long-range relationships and deeper historical information. By introducing memory-aware fractional-order updates, FOFedAvg improves communication efficiency and accelerates convergence while mitigating instability caused by heterogeneous, non-IID client data. We compare FOFedAvg against a broad set of established federated optimization algorithms on benchmark datasets including MNIST, FEMNIST, CIFAR-10, CIFAR-100, EMNIST, the Cleveland heart disease dataset, Sent140, PneumoniaMNIST, and Edge-IIoTset. Across a range of non-IID partitioning schemes, FOFedAvg is competitive with, and often outperforms, these baselines in terms of test performance and convergence speed. On the theoretical side, we prove that FOFedAvg converges to a stationary point under standard smoothness and bounded-variance assumptions for fractional order $0<α\le 1$. Together, these results show that fractional-order, memory-aware updates can substantially improve the robustness and effectiveness of federated learning, offering a practical path toward distributed training on heterogeneous data.
