Search papers, labs, and topics across Lattice.
This study conducts a multi-level analysis of bias in neural networks, examining the learned latent space, layer activations, and model parameters to uncover how biases propagate. It introduces three novel and extended bias detection methods鈥擲paceBias, ActivationBias, and WeightBias鈥攖hat offer insights into bias manifestations within the network architecture rather than merely evaluating outcomes. Experiments on gender classification and digit classification reveal that as training distributions balance, the internal disparity and detection performance decrease, underscoring the need for deeper understanding of AI model behavior.
Bias detection in neural networks can be revolutionized by analyzing latent spaces and activations, revealing how biases are embedded in model architecture itself.
This study investigates the presence and propagation of bias within Neural Networks through a comprehensive multi-level analysis spanning the learned latent space, layer activations, and the network's parameters. Based on this taxonomy, we propose three bias detection approaches: 1) SpaceBias (new method), which characterizes the latent space prior to the final classification layer using neighbor-probability distributions and quantifies bias with the two-sample Kolmogorov-Smirnov test on the per-group distributions. 2) ActivationBias (extension of the existing method InsideBias), which analyzes the activations of neural network filters and quantifies bias via a Mann-Whitney U test, based on the observed fact that underrepresented groups exhibit lower activation levels in the final convolutional layers. 3) WeightBias (extension of the existing method IFBiD), which uses a secondary neural network trained to identify biased patterns directly in the parameters of task-specific models. Unlike conventional methods, which assess neural network outcomes and treat the model as a black box, our proposed techniques provide insight into how biases manifest within the network architecture itself at different levels, offering a more nuanced and detailed understanding. Experiments are conducted on two complementary applications: gender classification in the DiveFace dataset (72,000 face images) and digit classification on a colored-MNIST benchmark with controlled bias severity. In total, more than 127,000 models with varying degrees and types of bias were trained and evaluated. The severity sweep shows that the internal disparity, and with it the detection performance, decreases smoothly as the training distribution approaches balance. The results highlight the importance of methods that provide deeper insight into the behavior of AI models.
