Search papers, labs, and topics across Lattice.
This paper introduces a synthetic data-driven framework for auditing biases in Human Action Recognition (HAR) models, focusing on the impact of visual identity attributes like skin color. They generate temporally consistent synthetic videos using the BEDLAM platform, enabling controlled interventions to isolate the effect of specific attributes on model predictions. Experiments on popular HAR models reveal statistically significant biases related to skin color, even when the action performed remains identical.
HAR models can exhibit statistically significant biases based on skin color, even when performing the exact same action.
Human Action Recognition (HAR) models are increasingly deployed in high-stakes environments, yet their fairness across different human appearances has not been analyzed. We introduce a framework for auditing bias in HAR models using synthetic video data, generated with full control over visual identity attributes such as skin color. Unlike prior work that focuses on static images or pose estimation, our approach preserves temporal consistency, allowing us to isolate and test how changes to a single attribute affect model predictions. Through controlled interventions using the BEDLAM simulation platform, we show whether some popular HAR models exhibit statistically significant biases on the skin color even when the motion remains identical. Our results highlight how models may encode unwanted visual associations, and we provide evidence of systematic errors across groups. This work contributes a framework for auditing HAR models and supports the development of more transparent, accountable systems in light of upcoming regulatory standards.
