Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Fairness Pruning, a targeted intervention method for identifying and mitigating demographic bias in large language models (LLMs) by analyzing differential activations in GLU-MLP layers. Using minimally contrastive prompt pairs, the authors successfully locate neurons that respond differently to demographic attributes, demonstrating that selectively zeroing these neurons can alter model responses to bias without significantly degrading performance. The results reveal a complex relationship between bias mitigation and model capabilities, where the intervention can lead to both reductions and destabilizations of bias, emphasizing the need for a nuanced approach to bias management in AI systems.
Zeroing just 40 neurons can retain 99.49% of a model's reasoning capabilities while altering its responses to demographic bias.
This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive prompt pairs and inference-time activation capture, the method identifies neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down_proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely surgical: zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.