Search papers, labs, and topics across Lattice.
This paper reveals a significant vulnerability in federated learning for language model fine-tuning, where a malicious parameter server can implant a privacy backdoor through a method called NeuroImprint. By assigning dedicated memorization neurons to each training sample, the attack allows for the covert extraction of sensitive training data without compromising the model's performance. The experiments show that this technique can reconstruct 59% to 79% of fine-tuning samples with high semantic fidelity across various language models and datasets, highlighting a critical privacy risk in current federated learning practices.
A malicious parameter server can exploit parameter-efficient fine-tuning to implant a privacy backdoor that reconstructs training data with alarming accuracy.
Federated learning (FL) enables multiple parties to collaboratively fine-tune language models for domain-specific tasks without sharing raw data. Since full model fine-tuning is often prohibitively expensive for FL clients, parameter-efficient fine-tuning (PEFT) has become the de facto approach in practice, freezing the base model and training only a small set of adapters. In this paper, we show that a malicious parameter server can stealthily corrupt a PEFT adapter into a privacy backdoor that implicitly memorizes the client's training samples as isolated per-sample parameter updates stored in separate neurons, without degrading model utility. Concretely, our attack, NeuroImprint, assigns a dedicated memorization neuron to each training sample and constrains that each neuron is updated at most once along the local fine-tuning trajectory. This design mitigates both cross-sample collisions and cross-step mixing introduced by large local batches and stateful optimizers (e.g., Adam/AdamW) in language-model fine-tuning. After fine-tuning, the resulting isolated per-sample updates can be analytically inverted in closed form to recover text embeddings, which are then deterministically mapped back to token sequences. To understand the generality of our method, we implemented NeuroImprint on multiple language models (BERT, GPT-2, Qwen2, and Llama3.2) and evaluated it across four fine-tuning datasets spanning diverse domains. The results demonstrate that our attack can reconstruct 59% to 79% of all finetuning samples with high semantic fidelity.
