Search papers, labs, and topics across Lattice.
This paper introduces a sparse subgradient algorithm tailored for Max-Plus neural networks, which replace standard arithmetic operations with maximum and summation. The key idea is to exploit the inherent sparsity in the subgradients of Max-Plus networks, where only neurons contributing to the maximum affect the loss. By minimizing the worst-sample loss and explicitly accounting for this algebraic sparsity during optimization, the proposed algorithm achieves more efficient updates.
Max-Plus networks, despite their interpretability, can be efficiently trained by exploiting the algebraic sparsity of their subgradients, leading to faster updates.
Deep Neural Networks are powerful tools for solving machine learning problems, but their training often involves dense and costly parameter updates. In this work, we use a novel Max-Plus neural architecture in which classical addition and multiplication are replaced with maximum and summation operations respectively. This is a promising architecture in terms of interpretability, but its training is challenging. A particular feature is that this algebraic structure naturally induces sparsity in the subgradients, as only neurons that contribute to the maximum affect the loss. However, standard backpropagation fails to exploit this sparsity, leading to unnecessary computations. In this work, we focus on the minimization of the worst sample loss which transfers this sparsity to the optimization loss. To address this, we propose a sparse subgradient algorithm that explicitly exploits the algebraic sparsity. By tailoring the optimization procedure to the non-smooth nature of Max-Plus models, our method achieves more efficient updates while retaining theoretical guarantees. This highlights a principled path toward bridging algebraic structure and scalable learning.
