Search papers, labs, and topics across Lattice.
This paper audits the provenance of labels in encrypted traffic classification benchmarks, revealing critical flaws in label assignment strategies across 14 datasets. The authors identify two main strategies: coarse inheritance, which risks mislabeling flows, and overstrict filtering, which may discard relevant data. Their analysis shows that no benchmark provides a clear pre-selection population, and discrepancies in label accuracy can significantly affect classifier performance, with balanced accuracy ceilings ranging from 0.56 to 0.76 depending on the strategy employed.
Mislabeling in encrypted traffic benchmarks can lead to a staggering 40% drop in classifier accuracy, challenging the validity of existing models.
Encrypted traffic classification infers semantics beyond the flow record from transport-layer observables, and supervised training rests on labels that hold for the individual flow they are attached to. Recent systematizations scrutinize model in- puts and data splits; we systematize the complementary label side. Across 14 audited benchmark entries, we identify two recurring label-side strategies: coarse inheritance, which risks labelling flows the evidence does not cover, and overstrict filtering, which keeps only self-attesting flows and risks dis- carding relevant ones. No audited entry exposes a countable pre-selection population, and the task objects downstream papers attach to the same labels disagree with the recovered record in 8 of 23 referenced cells. Under strict side-channel features we derive a representation-relative ceiling on bal- anced accuracy for any classifier restricted to those features: on the public benchmarks that inherit, it ranges from 0.56 to 0.76. On the filtering side, only 24.95% of connections in our fully captured corpus carry an observable SNI of their own; yet the discarded connections raise macro accuracy from 0.44 to 0.65 through same-run co-occurrence features. We end with recommendations for benchmark builders and users.
