Search papers, labs, and topics across Lattice.
This paper introduces GGSS, a novel method for debiasing generative vision-language models (VLMs) by steering visual tokens along geodesic arcs in a counterfactual bias subspace on the unit hypersphere. The approach effectively targets and corrects demographic biases in model outputs while maintaining high accuracy, outperforming ten existing debiasing baselines across various bias tests. GGSS achieves the lowest average bias on all evaluated models, demonstrating significant improvements in bias reduction without compromising performance on visual-language tasks.
GGSS reduces demographic bias in generative VLMs while preserving accuracy, outperforming traditional debiasing methods.
Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.
