Search papers, labs, and topics across Lattice.
This paper introduces Hide&Seek, an end-to-end differentiable model that addresses the challenges of instance-wise feature selection by jointly learning feature selection and prediction without information leakage. The method reformulates feature removal as a differentiable operation, allowing for a more stable training process through a parsimony-weight annealing framework. Experimental results demonstrate that Hide&Seek outperforms existing state-of-the-art models while being faster to train, highlighting its effectiveness in interpreting black-box models.
Instance-wise feature selection can now be performed without information leakage, enabling faster and more accurate predictions in black-box models.
Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.
