Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper analyzes the implicit regularization of gradient descent in deep matrix factorization when the target matrix is corrupted by noise. It derives spectral conditions on the target matrix, initialization, and step size that guarantee a low-rank phase during training in the noiseless setting. The analysis extends to the noisy case, providing convergence guarantees and quantifying the impact of noise on iteration complexity and eigenvalue recovery, demonstrating the persistence of the low-rank phase.
Even with noisy data, deep matrix factorization provably finds low-rank solutions under specific spectral conditions, offering insights into implicit regularization.
This paper studies the stability of low-rank implicit regularization in perturbed deep matrix factorization, where the target matrix is corrupted by a noise matrix. We first derive sufficient spectral conditions under which gradient descent exhibits a low-rank phase in the noiseless setting. These conditions show how the target spectrum, initialization, and step size jointly determine the existence of a nonempty low-rank interval. We then analyze the perturbed gradient descent dynamics, proving convergence guarantees and quantifying how the perturbation affects iteration complexity and eigenvalue recovery. Finally, we show that the low-rank phase persists under perturbation, with explicit dependence on the perturbation size. Numerical experiments support the theoretical findings.