Search papers, labs, and topics across Lattice.
This paper introduces CouCE, a unified causal framework designed to enhance Deep Metric Learning (DML) by addressing the dual challenges of background spurious correlations and foreground nuisance perturbations. By employing Orthogonal Dictionary-Based Backdoor Adjustment (ODBA) and Multi-Scale Randomized Causal Intervention (MSRCI), CouCE effectively neutralizes these confounders, leading to improved zero-shot generalization in DML models. Extensive evaluations on benchmark datasets reveal that CouCE outperforms existing methods, establishing a new state-of-the-art in debiased DML performance.
CouCE achieves state-of-the-art performance in debiased deep metric learning by simultaneously neutralizing both background and foreground confounders.
Deep Metric Learning (DML) often struggles with zero-shot generalization because standard objectives inherently capture what co-occurs rather than what causes similarity. Consequently, DML models are vulnerable to shortcut learning driven by two structurally distinct confounders: background spurious correlations (which create backdoor paths via scene context) and foreground nuisance perturbations (which inject non-semantic variations like pose or illumination). Although existing methods have proposed targeted solutions for each pathway individually, none can simultaneously address both due to their fundamentally distinct causal roles. To bridge this gap, we propose the Counterfactual Causal Embedding (CouCE), a unified causal framework that explicitly models and neutralizes both confounders. Specifically, we introduce Orthogonal Dictionary-Based Backdoor Adjustment (ODBA), which isolates spurious background patterns into a variance-gated dictionary and stably disentangles them from the learned embeddings via soft orthogonal regularization. Simultaneously, we propose Multi-Scale Randomized Causal Intervention (MSRCI) to enforce causal invariance against foreground nuisances through multi-scale Fourier amplitude randomization and a symmetric KL invariance constraint. Notably, CouCE seamlessly integrates with any proxy-based loss, incurring modest training overhead without requiring architectural modifications during inference. Extensive experiments on CUB-200-2011, Cars-196, and Stanford Online Products demonstrate that CouCE consistently achieves state-of-the-art performance, providing a principled and robust solution for debiased DML.
