2019 · 13 citations · 8 references
ComputingComputer ScienceHigh-performance ComputingCluster ComputingMachine LearningParallel ComputingAlgorithm AnalysisParallel Performance AnalysisPerformance ModelingHigh-performance Computing SystemsMachine Learning-based ForecastingNeural Network ModelingParallel Machine LearningPerformance PredictionPerformance Prediction ModelsPrediction Model DevelopmentPrediction Models
There are a lot of middle-class or small-class high-performance computing clusters at universities and research institutes, etc. Large volumes of job logs have been accumulated after many years of operation. In this paper, on the basis of accumulated job logs on a high-performance computing cluster, we examine and analyze the job logs. Then, we study machine learning based performance analysis and prediction methods for parallel jobs. Various machine learning methods such as multivariate linear fitting, artificial neural network are used to build performance prediction models. We compare the errors of each model, and select the optimal prediction model for different users. The experimental results show that we can obtain reasonable prediction accuracy using the selected machine learning algorithms.
8
Yan Zhai, Mingliang Liu, Jidong Zhai et al. · 2011 · 102 citations
Improving backfilling by using machine learning to predict running times
Éric Gaussier, David Glesser, Valentin Reis et al. · 2015 · 98 citations · Full text