IEEE Transactions on Cloud Computing · 2020 · 44 citations · 29 references
ComputingComputer ScienceComputer SystemsCloud InfrastructureCluster ComputingDistributed SystemsCloud Resource ManagementCompute Resource ManagementJob SchedulingPerformance Bottleneck AnalysisResource SchedulingWorkload SchedulingBottleneck EventsCloud Data CentersCluster SchedulingDisk I/o PerformanceOnline Job SchedulingPerformance Bottleneck DetectionResource Characterization
Workload characteristics are vital for both data center operation and job scheduling in co-located data centers, where online services and batch jobs are deployed on the same production cluster. In this article, a comprehensive analysis is conducted on Alibaba's cluster-trace-v2018 of a production cluster of 4034 machines. The findings and insights are the following: (1) The workload on the production cluster poses a daily cyclical fluctuation, in terms of CPU and disk I/O utilization, and the memory system has become the performance bottleneck of a co-located cluster. (2) Batch jobs including their tasks and derived instances can be approximated as Zipf distribution. However, for all batch jobs with directed acyclic graph dependency, they suffer from co-location with online services since the online services are highly prioritized. (3) The resource usages of containers have similar cyclical fluctuation consistent with the whole cluster, while their memory usages remain approximately constant. (4) The number of batch jobs co-located with online services is dependent on the mispredictions per kilo instructions of online services. In order to guarantee the QoS of online services, when the MPKI of online services rises, the number of batch jobs to be co-located on the same machine should decrease.
29
Eli Cortez, Anand Bonde, Alexandre Muzio et al. · 2017 · 592 citations
Workload characterization and prediction in the cloud: A multiple time series approach
Arijit Khan, Xifeng Yan, Tao Shu et al. · 2012 · 304 citations
Who limits the resource efficiency of my datacenter
Jing Jie Guo, Zihao Chang, Sa Wang et al. · 2019 · 262 citations