Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the interplay between grammatical gender and social semantic bias in contextual embeddings, particularly in gendered languages like Spanish, which has been largely overlooked in existing debiasing methods. By constructing balanced datasets from controlled templates and natural contexts, the authors develop a framework utilizing centroid estimators, SVM, and LDA to disentangle grammatical gender from semantic contamination. The findings indicate that unweighted controlled contexts provide the clearest grammatical gender direction, with the centroid estimator outperforming traditional discriminative approaches.
Unweighted controlled contexts yield the clearest grammatical gender direction, revealing that existing debiasing methods may overlook critical nuances in contextual embeddings.
Contextual language models conflate grammatical gender and social semantic bias in gendered languages such as Spanish. Existing gender debiasing approaches only operate on static word embeddings leaving contextual representations unexplored for this two dimensional gender disentanglement. To address the this issue, we make the first attempt to disentangle grammatical gender from semantic contamination for contextual embeddings. We construct both controlled templates and natural Wikipedia contexts to build balanced datasets of inanimate nouns, and design a framework equipped with centroid, Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) gender direction estimators as well as contamination-aware weighting strategies. A set of dual-objective evaluation metrics is proposed to balance the suppression of grammatical gender leakage on inanimate nouns and the preservation of semantic gender distinctions for occupation terms. The results reveal that unweighted controlled contexts yield the purest grammatical gender direction, and the centroid estimator achieves better performance than discriminative baselines.