Search papers, labs, and topics across Lattice.
This paper introduces qshap, a novel tool for performing Shapley decomposition of $R^2$ values specifically for gradient-boosted decision trees (GBDTs), enabling researchers to quantify feature-specific contributions to overall model performance. By decomposing the quadratic loss of individual observations, qshap allows for a more nuanced understanding of feature importance beyond traditional local attribution methods. The tool is compatible with popular GBDT frameworks such as xgboost, lightgbm, and catboost, and includes an optimized backend for oblivious trees that significantly enhances computational efficiency.
qshap reveals how individual features drive model performance, offering a deeper insight into GBDT effectiveness than traditional attribution methods.
Numerous methods have been developed to quantify feature attributions in individual predictions for tree ensembles. However, many applications require global measures of feature contributions to overall model performance. Although local attribution scores can be aggregated to characterize feature importance, such summaries do not directly decompose measures of predictive performance, such as $R^2$. This article introduces qshap, available in both R and Python, which provides Shapley decomposition of $R^2$ values for gradient-boosted decision trees (GBDTs) to quantify feature-specific contributions to model performance. By decomposing the quadratic loss of individual observations, qshap provides flexible tools to explore the importance of individual features and observations. qshap currently supports widely used GBDT implementations, including xgboost, lightgbm, and catboost, through a unified tree representation and efficient C++ backends. Its modular design can accommodate other GBDT implementations built from binary decision trees. In addition, we introduce a specialized backend for oblivious trees that exploits their symmetric structure to substantially accelerate computation.
