Search papers, labs, and topics across Lattice.
This paper addresses the limitations of variable importance scores generated by Random Forests (RF), which often undervalue correlated variables due to their interdependencies. By grouping variables based on their conditional correlations with the response variable, the authors propose two methods鈥攊ndividual grouping and clustering鈥攖o enhance the accuracy of variable importance assessments. Experimental results demonstrate that these approaches yield significant improvements in the reliability of variable importance scores, enabling better model interpretation and selection.
Correlated variables can be undervalued in Random Forests, but new grouping methods reveal their true importance, enhancing model insights.
Variable importance produced by Random Forests (RF) is used widely in statistical data analysis, and has played an important role in a variety of tasks such as assisting model interpretation, model selection and diagnosis, and cost-bounded learning etc. However, the calculation of variable importance in RF does not take into account of the correlations among variables, and variables that are correlated to many other variables tend to receive a lower importance index or being completely masked (i.e., with an importance index near zero) by other strongly correlated variables. To prevent influence from unwanted correlated variables in calculating variable importance, we propose to group variables by their conditional correlations (conditional on the response variable). We explore two computationally efficient options, with one grouping variables individually, and then separates the variable of interest from all correlated variables, while the other uses clustering to group variables according to their pair-wise conditional correlations. Our experiments show that both lead to sensible corrections to the importance of variables.
