Search papers, labs, and topics across Lattice.
This study conducts a mechanistic analysis of bi-encoder dense retrieval models to uncover the sources of gender bias, revealing that the bias originates in input embeddings and is propagated through specific late-layer attention heads. The authors implement targeted steering interventions, demonstrating that while embedding-level adjustments can neutralize score disparities, attention-level modifications lead to more nuanced directional shifts in bias. These insights not only clarify the mechanisms behind gender sensitivity in retrieval models but also present a foundation for effective debiasing strategies.
Steering interventions reveal that embedding adjustments neutralize gender bias, while attention modifications can strategically shift it, highlighting the complexity of relevance signals in retrieval models.
While gender bias in dense retrieval models is well documented, with prior work showing that models often score male-gendered documents higher than female or neutral variants, the internal mechanisms producing these disparities are poorly understood. In this paper, we mechanistically analyze bi-encoder models to localize gender sensitivity, finding that the signal originates in input embeddings and propagates through a small set of late-layer attention heads that carry both gender and term-matching signals. Guided by these findings, we test steering interventions at both identified points and find distinct effects: embedding-level steering non-specifically neutralizes score differences, while attention-level steering produces directional shifts. Our findings provide a mechanistic basis for targeted debiasing and highlight the challenge of disentangling gender from relevance signals in shared model components.
