SaRLog: Semantic-Aware Robust Log Anomaly Detection via BERT-Augmented Contrastive Learning

Abstract

Numerous deep learning-based methods have been developed to address the intricacies of anomaly detection tasks within system logs, presenting two significant challenges. First, balancing model complexity with the capacity to generate semantically meaningful representations for the downstream detection model, is a delicate task. Second, these methods generally depend on extensive labeled data for effective training. Despite efforts to address these challenges separately, a comprehensive solution that efficiently tackles both issues simultaneously are lacking. In response, we introduce Semantic-aware Robust Log (SaRLog), a comprehensive solution designed to overcome the limitations of existing methods by leveraging the contextual semantic information extraction capability of bidirectional encoder representations from transformers (BERTs) and the few-shot learning capability of the Siamese network. The Siamese network, featured with contractive loss, is implemented on top of a custom domain-specific fine-tuned BERT. Our comparative analysis validates SaRLog's effectiveness against established baseline methods, demonstrating F1 score improvement of up to 31.2% and 46.7% on BGL and Thunderbird data sets, respectively. Moreover, additional experimental analysis aimed at evaluating the few-shot learning capability highlights the robustness and generalization efficiency of SaRLog. Thus, by overcoming data set variability and improving model generalization, SaRLog advances log anomaly detection, thereby effectively handling complex log data challenges.