Search papers, labs, and topics across Lattice.
Fudan University, Shanghai Innovation Institute, OpenMOSS Team
2
0
3
Hybrid LLMs exhibit a "Seesaw Effect" in context extension, with linear attention models thriving on continual pretraining and sliding-window models excelling at extrapolation, a distinction rooted in their positional inductive biases.
Standard visual token pruning targets the wrong encoder depth: tapping into intermediate representations while asymmetrically skipping decoder sub-layers cuts MLLM inference FLOPs by 79% without requiring any retraining.
