Search papers, labs, and topics across Lattice.
HiFedProx replaces the standard quadratic proximal penalty of FedProx with a scale-matched power-type regularizer ($p \ge 2$) solved locally via same-minibatch Armijo backtracking. By providing a super-linear response to updates exceeding a calibrated reference displacement, the higher-order penalty strongly suppresses aberrant client drift while minimally restricting typical local optimization steps. On a heterogeneous FEMNIST benchmark under composite stress, intermediate powers ($p=6$ and $p=7$) reduce moderate- and severe-stress losses by 11.44% and 23.16% compared to standard FedProx ($p=2$), with performance declining at higher powers due to severe optimization curvature.
Standard quadratic regularization fails to contain runaway client drift under extreme heterogeneity, but tuning the proximal exponent to $p \in [5, 7]$ slashes severe-stress federated loss by over 23%.
Federated clients that perform several local optimization steps can return parameter displacements with widely different magnitudes. The quadratic regularization of FedProx grows linearly with displacement and therefore offers limited control over the contrast between ordinary and unusually large client movements. We here introduce HiFedProx, which replaces the quadratic penalty with a scale-matched power-type regularizer indexed by $p\geq2$. All powers have the same regularization-gradient magnitude at a reference displacement $R$, while every $p>2$ gives a weaker response below $R$ and a stronger response above it. An exact affine reference calculation shows that increasing $p$ compresses relative displacement disparities, although very large powers approach fixed-radius behavior and increase local curvature. HiFedProx combines this geometry with finite-budget stochastic client optimization and same-minibatch Armijo backtracking. In paired five-seed experiments on a frozen 60-writer FEMNIST subset, a common-parameter study over $p\in\{2,3,4,5,6,7,8\}$ shows similar clean-training performance but substantial gains under composite stress. The lowest moderate- and severe-stress losses occur at $p=7$ and $p=6$, improving over $p=2$ by $11.44\%$ and $23.16\%$, respectively. Although displacement-tail ratios continue to decrease through $p=8$, predictive performance peaks in an intermediate range and Armijo trial cost increases with $p$. These results indicate that the exponent should be calibrated rather than maximized. In our experiments, $p=5$--$7$ provides the most useful range.
