Search papers, labs, and topics across Lattice.
This paper introduces FedLNS, a novel framework that leverages normalization signatures to detect and mitigate malicious updates in federated learning for language models. By analyzing changes in trainable normalization-layer parameters, FedLNS screens client updates without requiring additional data exchange, thus preserving privacy while enhancing model integrity. Experimental results demonstrate that FedLNS outperforms six baseline methods in maintaining lower test perplexity even under significant target manipulation across various model architectures.
FedLNS effectively screens out malicious updates in federated learning, achieving superior model performance without compromising client privacy.
Federated training enables language models to learn from distributed private text, but the server cannot directly verify the local supervision or optimization process that produces each client update. A malicious client can therefore train on corrupted targets, introduce incorrect context-token associations, and degrade the global model through repeated aggregation. Such degradation can also increase the risk of unreliable or hallucinatory generation. We propose Federated Learning with Normalization Signatures (FedLNS), a server-side framework for lightweight malicious-update screening. FedLNS represents each client update through changes in trainable normalization-layer parameters and screens suspicious updates against a robust, history-aware cross-client reference. Because the signatures are extracted at the server from the returned local models, FedLNS requires no additional client-to-server parameter or metadata exchange compared to standard federated learning (FL) methods. After screening, the retained full-model updates can be aggregated using standard FL or another compatible aggregation rule. FedLNS requires no raw client data, trusted server dataset, labeled attack examples, or separately trained detector. Experiments on GPT-style, BERT-style, and LLaMA-style models trained from scratch with 200 clients show that, under 40% population-level target manipulation, FedLNS achieves lower test perplexity than the strongest of six baselines for all three architectures under both IID (independently and identically distributed) and non-IID data partitions.
