Search papers, labs, and topics across Lattice.
2
0
4
1
Positional encodings aren't just about position - they fundamentally prevent mean-field transformers from collapsing into meaningless single-point token distributions.
Despite its simplicity, mean pooling works surprisingly well because modern text encoders concentrate token embeddings, preserving crucial information about their distribution.
