Search papers, labs, and topics across Lattice.
The paper introduces De-paradox Tree, a recursive partitioning algorithm, to detect and interpret Simpson's paradox by uncovering subgroup patterns in observational data under assumed causal structures. The algorithm uses novel split criteria and balancing-based procedures to adjust for confounders and homogenize heterogeneous effects. Experiments demonstrate that De-paradox Tree constructs simpler, more interpretable trees, selects relevant covariates, and identifies nested opposite effects compared to existing methods, while also ensuring robust estimation of causal effects.
Uncover hidden subgroup patterns behind Simpson's paradox with De-paradox Tree, a new interpretable algorithm that helps non-experts make more reliable and informed decisions from complex observational data.
Real-world observational datasets and machine learning have revolutionized data-driven decision-making, yet many models rely on empirical associations that may be misleading due to confounding and subgroup heterogeneity. Simpson's paradox exemplifies this challenge, where aggregated and subgroup-level associations contradict each other, leading to misleading conclusions. Existing methods provide limited support for detecting and interpreting such paradoxical associations, especially for practitioners without deep causal expertise. We introduce De-paradox Tree, an interpretable algorithm designed to uncover hidden subgroup patterns behind paradoxical associations under assumed causal structures involving confounders and effect heterogeneity. It employs novel split criteria and balancing-based procedures to adjust for confounders and homogenize heterogeneous effects through recursive partitioning. Compared to state-of-the-art methods, De-paradox Tree builds simpler, more interpretable trees, selects relevant covariates, and identifies nested opposite effects while ensuring robust estimation of causal effects when causally admissible variables are provided. Our approach addresses the limitations of traditional causal inference and machine learning methods by introducing an interpretable framework that supports non-expert practitioners while explicitly acknowledging causal assumptions and scope limitations, enabling more reliable and informed decision-making in complex observational data environments.
