Search papers, labs, and topics across Lattice.
The paper introduces a novel evaluation protocol for IoT time-series anomaly detection (AD) that focuses on event-level performance under realistic data augmentations simulating real-world issues like sensor dropout, drift, and noise. They evaluate 14 representative AD models across seven datasets, including industrial datasets, using unified splits and event aggregation to provide a more practical benchmark. Their results show that no single model universally excels, with graph-based models performing best under dropout and long events, while other model families show sensitivity to specific augmentations like drift.
Forget point-level metrics: this benchmark reveals how IoT anomaly detection models crumble under realistic data perturbations like sensor dropout and drift, showing no universal winner exists.
Anomaly detection (AD) for safety-critical IoT time series should be judged at the event level: reliability and earliness under realistic perturbations. Yet many studies still emphasize point-level results on curated base datasets, limiting value for model selection in practice. We introduce an evaluation protocol with unified event-level augmentations that simulate real-world issues: calibrated sensor dropout, linear and log drift, additive noise, and window shifts. We also perform sensor-level probing via mask-as-missing zeroing with per-channel influence estimation to support root-cause analysis. We evaluate 14 representative models on five public anomaly datasets (SWaT, WADI, SMD, SKAB, TEP) and two industrial datasets (steam turbine, nuclear turbogenerator) using unified splits and event aggregation. There is no universal winner: graph-structured models transfer best under dropout and long events (e.g., on SWaT under additive noise F1 drops 0.804->0.677 for a graph autoencoder, 0.759->0.680 for a graph-attention variant, and 0.762->0.756 for a hybrid graph attention model); density/flow models work well on clean stationary plants but can be fragile to monotone drift; spectral CNNs lead when periodicity is strong; reconstruction autoencoders become competitive after basic sensor vetting; predictive/hybrid dynamics help when faults break temporal dependencies but remain window-sensitive. The protocol also informs design choices: on SWaT under log drift, replacing normalizing flows with Gaussian density reduces high-stress F1 from ~0.75 to ~0.57, and fixing a learned DAG gives a small clean-set gain (~0.5-1.0 points) but increases drift sensitivity by ~8x.
