Parikshit Ram

Learned expand-and-sparsify representations with ReLU activation match dense layers at significantly reduced sparsity, and binarization costs learned sparse codes nothing.

Achieving perfect length generalization with a transformer model of just 280 parameters could revolutionize how we approach algorithmic tasks in neural networks.