Search papers, labs, and topics across Lattice.
This paper introduces a behavioral characterization framework for evaluating intrusion detection in Industrial Control Systems (ICS) by mapping raw multivariate process traces into five interpretable physical primitives: drift, spike, oscillation, repetition, and switching. By applying this framework to three public ICS benchmarks (SWaT, WADI, and HAI), the authors reveal significant behavioral shifts during attack windows and highlight the distinct behavioral characteristics of each dataset, indicating both cross-dataset bias and intra-dataset diversity. The findings demonstrate that traditional binary evaluation metrics can obscure critical performance insights, with macro F1 scores dropping significantly when transitioning from binary to behavior-proxy multiclass predictions, underscoring the need for behavior-stratified evaluation methods.
Conventional binary metrics can mask critical performance insights, with macro F1 scores plummeting from 85.44% to 37.84% when using behavior-proxy multiclass predictions.
Intrusion detection in Industrial Control Systems (ICS) is typically evaluated on a small set of public benchmarks using binary ``normal''versus ``attack''labels, a practice that can mask the behavioral diversity of cyber-physical attacks. To address this limitation, we propose a behavioral characterization framework that maps raw multivariate process traces into five interpretable physical primitives: drift, spike, oscillation, repetition, and switching. We apply the framework to three widely used ICS benchmarks, namely, SWaT, WADI, and HAI, and show that attack windows exhibit clear behavioral shifts relative to normal operation while the three datasets occupy largely distinct regions of the behavioral space, revealing both cross-dataset bias and intra-dataset diversity. In particular, WADI is dominated by repetition, HAI emphasizes sustained drift and oscillation, and SWaT is characterized by stealthier frozen-telemetry behavior. To examine the evaluation implications, we use an indicative Random Forest baseline and show that aggregate binary metrics can limit visibility into performance across different behavioral proxies. For example, in SWaT, macro F1 drops from 85.44% under binary evaluation to 37.84% under behavior-proxy multiclass prediction, with similar degradations observed on WADI and HAI. Based on these findings, we argue for complementing conventional binary benchmarking with behavior-stratified evaluation to expose blind spots that aggregate scores leave hidden and to better support targeted incident response.
