Search papers, labs, and topics across Lattice.
This paper introduces Deflation-Free Sparse Optimal Scoring (DFSOS), a novel method for linear discriminant analysis that estimates all discriminant vectors simultaneously under a global orthogonality constraint, avoiding error propagation inherent in deflation-based approaches. DFSOS combines Bregman iteration with orthogonality-constrained optimization, decomposing the problem into tractable subproblems. Experiments on synthetic and real-world time series data show that DFSOS achieves classification accuracy comparable to or better than existing deflation-based methods, demonstrating the effectiveness of a deflation-free framework.
Ditch deflation: A new sparse discriminant analysis method sidesteps error propagation and achieves state-of-the-art accuracy by estimating all discriminant vectors simultaneously.
Sparse Optimal Scoring (SOS) reformulates linear discriminant analysis to enable feature selection through elastic net regularization, making it well-suited for high-dimensional settings where the number of features exceeds observations. Most existing SOS methods use deflation-based strategies that compute discriminant vectors sequentially, which can propagate errors and produce suboptimal solutions. We propose a novel approach that estimates all discriminant vectors simultaneously under an explicit global orthogonality constraint, which we call Deflation-Free Sparse Optimal Scoring (DFSOS). DFSOS combines Bregman iteration with orthogonality-constrained optimization, decomposing the problem into tractable subproblems for scoring vectors, discriminant vectors, and orthogonality enforcement. We establish convergence to stationary points of the augmented Lagrangian under mild conditions. Extensive experiments using synthetic data and real-world time series data demonstrate that DFSOS achieves classification accuracy comparable to or better than existing deflation-based methods. These results indicate that deflation-free approaches offer a robust and effective framework for sparse discriminant analysis in high-dimensional problems.
