Search papers, labs, and topics across Lattice.
This paper introduces a novel method for out-of-distribution (OOD) detection using sparse autoencoders (SAEs) to extract interpretable features from intermediate neural network activations. By analyzing the distinct sparse feature sets activated by in-distribution (ID) versus OOD samples, the authors derive a new OOD score based on cosine similarity between test sample activations and mean ID class activations. The approach achieves state-of-the-art performance on OOD detection benchmarks while providing valuable insights into the impact of distribution shifts on learned representations.
Distinct sparse features from intermediate layers reveal how neural networks misclassify out-of-distribution samples, leading to a new state-of-the-art detection method.
Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models. Neural networks often produce overconfident predictions for inputs that deviate from their training data, leading to significant degradation in performance. While many OOD detection methods focus on the final output layer, they neglect the rich hierarchical information present in intermediate network layers. This paper introduces a novel approach that leverages sparse autoencoders (SAEs) to learn interpretable features from these intermediate activations. We find that in-distribution (ID) and OOD data activate distinct sets of these sparse features. We propose a new OOD score derived from the cosine similarity between the sparse feature activations of a test sample and the mean activations of ID classes. Our post-hoc detection method not only achieves state-of-the-art performance on standard OOD detection benchmarks, but yields interpretable insights into how distribution shift affects learned representations.
