Search papers, labs, and topics across Lattice.
This paper benchmarks static word embeddings (Word2Vec, GloVe, FastText) against BERT for log anomaly detection, revealing a trade-off between effectiveness and efficiency. To address this, they introduce QTyBERT, a quantized BERT variant enhanced with cross-system embedding, CroSysEh, trained unsupervisedly to improve semantic expressiveness. Experiments on three datasets show QTyBERT achieves comparable or better detection performance than BERT with significantly improved embedding generation time, approaching the efficiency of static embeddings.
Quantizing BERT and training it across multiple systems lets you achieve anomaly detection performance on par with full BERT, but with the speed of static word embeddings.
Recent deep learning (DL) methods for log anomaly detection increasingly rely on semantic log representation methods that convert the textual content of log events into vector embeddings as input to DL models. However, these DL methods are typically evaluated as end-to-end pipelines, while the impact of different semantic representation methods is not well understood. In this paper, we benchmark widely used semantic log representation methods, including static word embedding methods (Word2Vec, GloVe, and FastText) and the BERT-based contextual embedding method, across diverse DL models for log-event level anomaly detection on three publicly available log datasets: BGL, Thunderbird, and Spirit. We identify an effectiveness--efficiency trade off under CPU deployment settings: the BERT-based method is more effective, but incurs substantially longer log embedding generation time, limiting its practicality; static word embedding methods are efficient but are generally less effective and may yield insufficient detection performance. Motivated by this finding, we propose QTyBERT, a novel semantic log representation method that better balances this trade-off. QTyBERT uses SysBE, a lightweight BERT variant with system-specific quantization, to efficiently encode log events into vector embeddings on CPUs, and leverages CroSysEh to enhance the semantic expressiveness of these log embeddings. CroSysEh is trained unsupervisedly using unlabeled logs from multiple systems to capture the underlying semantic structure of the BERT model's embedding space. We evaluate QTyBERT against existing semantic log representation methods. Our results show that, for the DL models, using QTyBERT-generated log embeddings achieves detection effectiveness comparable to or better than BERT-generated log embeddings, while bringing log embedding generation time closer to that of static word embedding methods.
