Chen Chen

X. Yan, C. Chen, and X. Li are with Department of Automation, Tsinghua University

Papers on Lattice

Total citations

Topics

h-index

Publication activitypapers/week, last 8 weeks

Research focus

Architecture Design (Transformers, SSMs, MoE) (1)Distributed Systems & Hardware (1)Training Efficiency & Optimization (1)

Frequent co-authors

Lianghui Zhu (1)Yuxin Fang (1)Bencheng Liao (1)Shijie Wang (1)

Papers (1)

Mar 16, 2026

Tsinghua AI2w ago·also ByteDance, SJTU

Mixture-of-Depths Attention

LLMs can now scale depth more effectively: a new attention mechanism recovers diluted features in deeper layers, boosting performance with negligible overhead.

Lianghui Zhu, Yuxin Fang, Bencheng Liao +10

Architecture Design (Transformers, SSMs, MoE)Distributed Systems & Hardware Training Efficiency & Optimization

Search

Chen Chen

Publication activitypapers/week, last 8 weeks

Research focus

Frequent co-authors

Papers (1)