Search papers, labs, and topics across Lattice.
This paper provides a systematic analysis of error sources in global feature effect estimation for black-box models, focusing on Partial Dependence (PD) and Accumulated Local Effects (ALE) plots. They derive a mean-squared-error decomposition to disentangle model bias, estimation bias, model variance, and estimation variance, analyzing their dependence on model characteristics, data selection (training vs. holdout), and sample size. Through simulations, they find that while holdout data is theoretically cleaner, using training data is often preferable due to the larger sample size, and cross-validation can reduce model variance, especially for overfitting models.
Forget holdout data for feature effect estimation: training data's larger sample size usually wins, and cross-validation can further reduce model variance.
Global feature effects such as PD and ALE plots are widely used to interpret black-box models. However, they are only estimates of true underlying effects, and their reliability depends on multiple sources of error. Despite the popularity of global feature effects, these error sources are largely unexplored. In particular, the practically relevant question of whether to use training or holdout data to estimate feature effects remains unanswered. We address this gap by providing a systematic, estimator-level analysis that disentangles sources of bias and variance for PD and ALE. To this end, we derive a mean-squared-error decomposition that separates model bias, estimation bias, model variance, and estimation variance, and analyze their dependence on model characteristics, data selection, and sample size. We validate our theoretical findings through an extensive simulation study across multiple data-generating processes, learners, estimation strategies (training data, validation data, and cross-validation), and sample sizes. Our results reveal that, while using holdout data is theoretically the cleanest, potential biases arising from the training data are empirically negligible and dominated by the impact of the usually higher sample size. The estimation variance depends on both the presence of interactions and the sample size, with ALE being particularly sensitive to the latter. Cross-validation-based estimation is a promising approach that reduces the model variance component, particularly for overfitting models. Our analysis provides a principled explanation of the sources of error in feature effect estimates and offers concrete guidance on choosing estimation strategies when interpreting machine learning models.
