Search papers, labs, and topics across Lattice.
This study explores feature evolution in Vision Transformers (ViTs) by visualizing training dynamics across network depth and epochs using Sparse Autoencoders (SAEs) to extract and analyze candidate sparse features. The findings reveal that feature migration is predominantly concentrated in the early training stages, with a tendency for features to be more detectable in earlier layers, and that deeper layers stabilize more quickly than shallower ones. This framework not only enhances our understanding of ViT learning processes but also provides insights into the temporal dynamics of feature organization during training.
Feature migration in Vision Transformers is predominantly an early-stage phenomenon, with deeper layers stabilizing faster than shallow ones.
We present a novel view on feature evolution in Vision Transformers (ViTs) by visualizing the training process over two dimensions -- network depth (layer) and training time (epochs). We employ Sparse Autoencoders (SAEs) to extract candidate sparse features from CLS-token representations and compare their activation profiles across epoch--layer pairs. This allows us to study feature-level dynamics that are not directly visible from representation-level similarity measures. Furthermore, we demonstrate how this framework of feature evolution allows us to describe feature migration, the change in the layer where a feature is most detectable during training. Our experiments show that migration is concentrated early in training, occurs more often toward earlier layers than toward deeper layers, and declines as feature organization stabilizes. We further find that deeper layers stabilize earlier and more strongly than shallow layers. The results show that our approach can be employed as a tool for understanding how ViTs learn and evolve.
