Search papers, labs, and topics across Lattice.
This study evaluates the adversarial robustness and explainability stability of cybersecurity classifiers, specifically Random Forest and XGBoost, against five different attack methods across four tabular datasets. The introduction of the Explainability Stability Index (ESI) reveals that while gradient-based black-box attacks yield misleadingly high robustness scores for XGBoost, they still induce significant attribution drift, highlighting a critical disconnect between prediction stability and explanation reliability. The findings underscore the necessity of jointly measuring robustness and explainability to better inform security analysts in their decision-making processes.
Gradient-based attacks can mislead by suggesting high robustness in XGBoost classifiers, while actually destabilizing their explanations.
Adversarial attacks on cybersecurity classifiers pose a dual threat: degrading predictions and destabilising the SHAP-based explanations that security analysts rely on to understand and triage alerts. We extend our prior MLP conference study to Random Forest and XGBoost across four tabular security datasets (phishing URLs, UNSW-NB15, NF-ToN-IoT, HIKARI-2021), evaluating five attacks including three black-box methods applicable to non-differentiable tree models. We introduce the Explainability Stability Index (ESI), a scalar metric computed from TreeSHAP attribution drift under adversarial perturbation, reported on the same [0,1] scale as the Robustness Index (RI). A key finding is that gradient-based black-box attacks (ZOO) produce degenerate results against XGBoost (apparent RI ~0.98) due to piecewise-constant prediction surfaces, while score-based Square Attack reveals genuine vulnerability (RI ~0.36). These degenerate perturbations still drive substantial attribution drift: XGBoost ESI ~0.06-0.16 despite near-perfect ZOO robustness, versus 0.14-0.29 for RF, showing that prediction robustness and explanation stability are distinct axes requiring joint measurement. A two-axis framework (gradient dependence, query efficiency) explains the observed attack ranking and yields practical guidance for tree ensemble evaluation. A step-size ablation explains a counterintuitive PGD anomaly on z-score normalised tabular data.
