Search papers, labs, and topics across Lattice.
This paper draws an analogy between a transformer layer's operations (projections and layer normalization) and a step of the power method. It demonstrates that tokens passing through a layer tend to align with the principal eigenvector of the product of the layer's output and value weight matrices. This alignment is particularly pronounced in transformers with shared weights, both empirically and analytically, and can be exploited to steer the transformer's output.
Transformers secretly act like the power method, concentrating token embeddings along a dominant eigenvector with each layer.
In the paper we show that there is an analogy between the operations occurring in a layer of a transformer (projections and layer normalizations, disregarding the feedforward neural network) and a step in the power method. Coherently with this analogy, we show that passing through a layer the tokens tend to be tilted towards the principal eigenvector of a matrix which is the product of the output and value weight matrices of that layer. In the special case of a transformer with shared weights (i.e., in which all layers have identical weights) then the alignment with this principal eigenvector is particularly evident empirically, and can also be shown analytically. The analogy also suggests a method to steer the output of the transformer towards an arbitrary desired direction in token space.
