Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors exploit the classic "dying ReLU" failure mode to engineer training-time availability attacks via dynamic data reordering and gradient-inversion poisoning. By strategically pushing pre-activations into negative regimes, these attacks induce widespread gradient starvation to permanently degrade network capacity without requiring label flips or direct weight modifications. On MNIST, reordering just 100 clean training examples measurably reduces test accuracy, while injecting 200 synthesized class-conditioned poisoned samples drops performance from 96% down to 86–88% within five epochs.
An adversary can permanently cripple deep ReLU networks without ever altering model parameters or labels, weaponizing the "dying neuron" pathology through subtle batch reordering and gradient-inversion poisoning.
Rectified linear unit (ReLU) networks can suffer from dying neurons, where units with persistently negative pre-activations produce zero outputs, blocking gradients through their activations. To exploit this failure mode, we present three training-time availability attacks based on data ordering and poisoning. We begin with the basic dynamic data-ordering attack (DOA), which greedily constructs a training prefix by selecting the next example that minimizes the target layer's post-update weight sum, aiming to push ReLU units toward negative pre-activations without modifying training samples or labels. We then develop two poisoning attacks, IG-DOA and IG-SKA, which use gradient inversion to synthesize class-conditioned samples by matching reference gradients in adverse model states constructed through data ordering or soft knockout, respectively. Soft knockout rearranges weights across adjacent layers to concentrate negative contributions. On a fully connected ReLU network trained on MNIST, ordering 100 of 60,000 training examples reduces test accuracy from 96% to 95% after only five epochs. Adding 200 poisoned samples from a single class reduces test accuracy to approximately 86-88% after five epochs in most evaluated conditions, compared with approximately 96% under clean training. These results demonstrate that ReLU-targeted data ordering and poisoning can impair learning without directly modifying the victim model's parameters.