2020 · 18 citations · 16 references
ComputingComputer ScienceFeature EngineeringMachine LearningNatural Language ProcessingAnomaly DetectionDeep LearningLog AnalysisMessage LoggingRepresentation LearningText Feature EngineeringAnomaly Detection In LogsAutoencoder-based Anomaly DetectionClass ImbalanceImbalanced ClassificationLog ClassificationLog Data AnalysisMachine Learning For Log AnalysisText Anomaly Detection
Imbalanced data is a significant challenge in classification with machine learning algorithms. This is particularly important with log message data as negative logs are sparse so this data is typically imbalanced. In this paper, a model to generate text log messages is proposed which employs a SeqGAN network. An Autoencoder is used for feature extraction and anomaly detection is done using a GRU network. The proposed model is evaluated with three imbalanced log data sets, namely BGL, OpenStack, and Thunderbird. Results are presented which show that appropriate oversampling and data balancing improves anomaly detection accuracy.
16
SMOTE: Synthetic Minority Over-sampling Technique
Nitesh V. Chawla, Kevin W. Bowyer, Lawrence Hall et al. · Journal of Artificial Intelligence Research · 2002 · 31.2K citations · Full text