Human-centric Computing and Information Sciences · 2019 · 28 citations · 19 references
ComputingComputer ScienceComputer SystemsDistributed SystemsParallel ComputingPredictive AnalyticsDistributed ComputingDistributed Data ProcessingPerformance ModelingStream ProcessingStreaming AlgorithmsData Stream ProcessingDistributed Stream ProcessingGaussian Process RegressionMetricsPerformance PredictionPerformance Prediction ModelsRidge RegressionStream Processing Metrics
Abstract Worldwide sensor streams are expanding continuously with unbounded velocity in volume, and for this acceleration, there is an adaptation of large stream data processing system from the homogeneous to rack-scale architecture which makes serious concern in the domain of workload optimization, scheduling, and resource management algorithms. Our proposed framework is based on providing architecture independent performance prediction model to enable resource adaptive distributed stream data processing platform. It is comprised of seven pre-defined domain for dynamic data stream metrics including a self-driven model which tries to fit these metrics using ridge regularization regression algorithm. Another significant contribution lies in fully-automated performance prediction model inherited from the state-of-the-art distributed data management system for distributed stream processing systems using Gaussian processes regression that cluster metrics with the help of dimensionality reduction algorithm. We implemented its base on Apache Heron and evaluated with proposed Benchmark Suite comprising of five domain-specific topologies. To assess the proposed methodologies, we forcefully ingest tuple skewness among the benchmarking topologies to set up the ground truth for predictions and found that accuracy of predicting the performance of data streams increased up to 80.62% from 66.36% along with the reduction of error from 37.14 to 16.06%.
19
Ankit Toshniwal, Siddarth Taneja, Amit K. Shukla et al. · 2014 · 858 citations
Brendan Burns, Brian Grant, David Oppenheimer et al. · Communications of the ACM · 2016 · 573 citations · Full text
Automatic Database Management System Tuning Through Large-scale Machine Learning
Dana Van Aken, Andrew Pavlo, Geoffrey J. Gordon et al. · 2017 · 550 citations · Full text
Sanjeev Kulkarni, Nikunj Bhagat, Maosong Fu et al. · 2015 · 547 citations · Full text