Search papers, labs, and topics across Lattice.
University of New South Wales University of Texas, Austin City University of Hong Kong
2
0
2
Hyper-connections create a fading memory in Transformers, with the effective context horizon directly tied to the second singular value of the stream-mixing matrices.
Even when an input activates only $k$ of $s$ hidden units, sample complexity cannot escape the total parameter count, incurring an inescapable $\tilde{\Theta}(\sqrt{sk/m})$ rate while eliminating all explicit dependencies on input dimension.
