Search papers, labs, and topics across Lattice.
This paper introduces a novel method for Out-of-Distribution (OOD) detection in Vision Transformers (ViTs) by analyzing internal representations through Singular Value Decomposition (SVD) of affine layer weight matrices. By projecting activations onto leading right singular vectors, the authors create layer-intrinsic representations that are used to compute class-conditional typicality scores, which are then aggregated into two-dimensional typicality maps. The resulting Prototype Alignment Score (PAS) and Multi-Layer Soft Voting (MLSV) score demonstrate competitive OOD detection performance on ViT-B/16 fine-tuned on CIFAR-100, without the need for retraining or OOD exposure.
Class-conditional typicality maps derived from SVD can significantly enhance OOD detection in Vision Transformers without requiring additional training.
We present a method for analyzing the internal representations of Vision Transformers (ViTs) exploiting the geometry of their learned parameters. Each affine layer's weight matrix is factored via Singular Value Decomposition (SVD), and activations are projected onto the leading right singular vectors to obtain compact, layer-intrinsic representations. A class-conditional density model is then fitted at each layer, producing per-class \emph{typicality scores} that are stacked across depth into \emph{typicality maps}: two-dimensional summaries of how class-specific evidence evolves through the network. From these maps, we derive two post-hoc scores for Out-Of-Distribution (OOD) detection: a \emph{Prototype Alignment Score} (PAS), measuring agreement with class reference prototype patterns, and a \emph{Multi-Layer Soft Voting} (MLSV) score, capturing cross-layer consensus without stored prototypes. On ViT-B/16 fine-tuned on CIFAR-100, the proposed scores achieve competitive detection performance without retraining or OOD exposure.
