Search papers, labs, and topics across Lattice.
This paper extends the LASSO-Clip-EN (LCEN) algorithm, previously used for regression, to classification tasks while preserving its interpretability. The modified LCEN is evaluated on binary and multiclass datasets, demonstrating competitive performance in macro F1 and MCC compared to other models, while also achieving feature sparsity. Additionally, the paper introduces and evaluates a weighted focal differentiable MCC (diffMCC) loss function, showing it outperforms weighted cross-entropy loss in these classification tasks.
Forget cross-entropy: a differentiable MCC loss function can boost your classification accuracy by nearly 5% on F1 score and 8.5% on MCC.
The LASSO-Clip-EN (LCEN) algorithm was previously introduced for nonlinear, interpretable feature selection and machine learning. However, its design and use was limited to regression tasks. In this work, we create a modified version of the LCEN algorithm that is suitable for classification tasks and maintains its desirable properties, such as interpretability. This modified LCEN algorithm is evaluated on four widely used binary and multiclass classification datasets. In these experiments, LCEN is compared against 10 other model types and consistently reaches high test-set macro F$_1$ score and Matthews correlation coefficient (MCC) metrics, higher than that of the majority of investigated models. LCEN models for classification remain sparse, eliminating an average of 56% of all input features in the experiments performed. Furthermore, LCEN-selected features are used to retrain all models using the same data, leading to statistically significant performance improvements in three of the experiments and insignificant differences in the fourth when compared to using all features or other feature selection methods. Simultaneously, the weighted focal differentiable MCC (diffMCC) loss function is evaluated on the same datasets. Models trained with the diffMCC loss function are always the best-performing methods in these experiments, and reach test-set macro F$_1$ scores that are, on average, 4.9% higher and MCCs that are 8.5% higher than those obtained by models trained with the weighted cross-entropy loss. These results highlight the performance of LCEN as a feature selection and machine learning algorithm also for classification tasks, and how the diffMCC loss function can train very accurate models, surpassing the weighted cross-entropy loss in the tasks investigated.
