Search papers, labs, and topics across Lattice.
This study investigates the concept of Super Weights in large language models (LLMs), revealing that while these parameters significantly impact performance when removed, training them in isolation leads to drastic accuracy drops, even to random-guessing levels. The authors demonstrate that targeting Super Weights specifically does not yield improvements, contrasting with the success of training randomly chosen parameters in the same layers. Their findings suggest that effective fine-tuning requires a holistic approach to parameter updates rather than focusing on individual weights deemed important.
Targeting Super Weights in LLMs can lead to performance collapse, challenging assumptions about parameter importance and trainability.
Recent work identified Super Weights, individual parameters whose removal degrades model performance by orders of magnitude. We show that this degradation due to pruning Super Weights does not universally apply to all LLMs. Furthermore, if these parameters are so important, Super Weight-aware training should be effective. We show the opposite. Training Super Weights in isolation (100 to 8,192 parameters) drops accuracy to random-guessing levels on both OLMo-1B and OLMo-7B, and expanding to local neighborhoods of up to 36K parameters provides no improvement. The failure is specific to Super Weight coordinates: training an equal number of randomly chosen positions in the same down_proj layers instead improves over the baseline, so the collapse comes from targeting Super Weights, not from sparsity itself. Vanilla LoRA, updating every position in attention weight matrices through low-rank structure, succeeds with only 0.16% of parameters, and applying the same low-rank update to down_proj succeeds as well. A 10-seed ablation confirms that constraining LoRA updates at positions corresponding to Super Weight coordinates yields statistically indistinguishable results. These findings establish that parameter importance does not imply parameter trainability in isolation, and that effective fine-tuning relies on structured decompositions over entire layers rather than targeting individually important weights.
Amazon Science