Search papers, labs, and topics across Lattice.
This paper analyzes the convergence of two-layer ReLU neural networks trained with Gaussian randomly masked inputs, a scenario relevant to Gaussian dropout, noisy input training, and federated learning. Using Neural Tangent Kernel (NTK) theory, the authors prove linear convergence up to an error region that scales with the variance of the Gaussian mask. A key technical advance involves resolving the randomness introduced by the non-linear activation function under Gaussian masking.
Gaussian input masking, common in federated learning and noisy sensor networks, provably converges for two-layer ReLU networks under an NTK regime, with explicit error bounds tied to the mask variance.
We investigate the convergence guarantee of two-layer neural network training with Gaussian randomly masked inputs. This scenario corresponds to Gaussian dropout at the input level, or noisy input training common in sensor networks, privacy-preserving training, and federated learning, where each user may have access to partial or corrupted features. Using a Neural Tangent Kernel (NTK) analysis, we demonstrate that training a two-layer ReLU network with Gaussian randomly masked inputs achieves linear convergence up to an error region proportional to the mask's variance. A key technical contribution is resolving the randomness within the non-linear activation, a problem of independent interest.
