Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a Newton-type algorithm for Nonnegative Matrix Factorization (NMF) that utilizes the Kullback-Leibler divergence, particularly suited for count data like term-document matrices. By leveraging the second-order Taylor expansion of the loss function, the authors propose a generalization of the HALS algorithm to efficiently minimize a non-separable surrogate, resulting in a method that demonstrates provable convergence. The proposed KL-NMF algorithm outperforms existing state-of-the-art methods across diverse datasets, highlighting its effectiveness in unsupervised learning tasks.
A novel Newton-type approach to KL-NMF achieves superior performance and provable convergence, challenging the limitations of existing first-order methods.
Nonnegative Matrix Factorization (NMF) is a fundamental tool in unsupervised learning, which approximates a nonnegative matrix by the product of two low-rank nonnegative factors. The Kullback-Leibler (KL) divergence is best suited to measure the data to model discrepancy when the decomposed data sample follows a Poisson distribution, which is the case for count datasets such as term-document matrices or images. Most KL-NMF algorithms in the literature minimize a separable majorant of the loss to find their next iterate. We argue that this method has reached its limits and propose to use instead the second-order Taylor expansion of the loss, leading to a Newton-type method. We minimize this non-separable surrogate by proposing a generalization of the well-known HALS algorithm. This yields an efficient KL-NMF algorithm which provably converges and which competes favorably with state-of-the-art algorithms on a large variety of datasets.