Luke McDermott

UC San Diego

Papers on Lattice

Total citations

Topics

Publication activitypapers/week, last 8 weeks

Research focus

Architecture Design (Transformers, SSMs, MoE) (1)Scaling Laws & Emergent Abilities (1)

Frequent co-authors

Robert W. Heath (1)Rahul Parhi (1)

Papers (1)

Jun 24, 2026

2w ago

Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention

Parametric attention mechanisms could be the key to unlocking lifelong learning in transformers, but they still struggle with memory limitations and costly updates.

Luke McDermott, Robert W. Heath, Rahul Parhi

Architecture Design (Transformers, SSMs, MoE)Scaling Laws & Emergent Abilities

Search

Luke McDermott

Publication activitypapers/week, last 8 weeks

Research focus

Frequent co-authors

Papers (1)