Search papers, labs, and topics across Lattice.
This paper introduces a joint model and data sparsification method using a Bayesian approach based on automatic relevance determination (ARD). By simultaneously learning feature and sample relevancies via marginal likelihood optimization, the method extends traditional ARD to handle data contaminations and outliers more effectively. Experiments on regression tasks demonstrate that this joint ARD approach produces sparse and robust prediction models.
Outliers got you down? This Bayesian method prunes both features *and* data points to build robust sparse models, all in one shot.
Sparse recovery in linear systems underpins applications from signal processing to high-dimensional regression. Sparse Bayesian Learning, grounded in the principle of automatic relevance determination (ARD), offers a practical Bayesian mechanism for feature sparsity via marginal likelihood optimization. Yet, its reliance on a homoscedastic noise model renders it sensitive to data contaminations such as outliers or misspecified noise, harming model fit and predictions. Instead, we propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification via a single Bayesian objective. This symmetric pruning of model and data offers a natural extension that preserves conjugacy, admits closed-form updates for standard optimization procedures, and aligns with perspectives from robust regression and influence functions. Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
