Alejandro Francisco Queiruga

Papers on Lattice

Total citations

Topics

Publication activitypapers/week, last 8 weeks

Research focus

Architecture Design (Transformers, SSMs, MoE) (1)Scaling Laws & Emergent Abilities (1)Training Efficiency & Optimization (1)

Papers (1)

Feb 16, 2026

Alejandro Francisco QueirugaFeb 16, 2026

Divine Benevolence is an $x^2$: GLUs scale asymptotically faster than MLPs

GLUs' secret weapon is out: their piecewise quadratic nature allows them to scale asymptotically faster than MLPs (L(P) ∝ P⁻³ vs. P⁻²), potentially revolutionizing architecture design.

Alejandro Francisco Queiruga

Architecture Design (Transformers, SSMs, MoE)Scaling Laws & Emergent Abilities Training Efficiency & Optimization

Search

Alejandro Francisco Queiruga

Publication activitypapers/week, last 8 weeks

Research focus

Papers (1)