Science and Technology for Energy Transition · 2024 · 13 citations · 30 references
ComputingCloud ComputingCloud ArchitectureCloud InfrastructureEdge ComputingCloud Resource ManagementEdge-cloud ArchitectureRenewable EnergyResource ProvisioningResource SchedulingWorkload SchedulingCarbon-aware SchedulingCloud SchedulingDistributed Machine LearningEdge-cloud ContinuumEdge-to-cloud ContinuumEnergy-aware SchedulingGeo-distributed SystemsPower Consumption
As an increasing number of Distributed Machine Learning (DML) tasks are hosted on cloud platforms in the edge-cloud continuum, Data Centers (DCs) with massive data and computational requirements have become one of the world’s largest energy consumers, leading to significant carbon emissions. Reducing energy consumption and carbon emissions is an extremely crucial and challenging issue for the sustainable development of cloud service providers. While utilizing renewable energy can help reduce the carbon emissions of DCs, the intermittent and unstable nature still causes DCs to rely heavily on high-carbon brown energy. For the resource-intensive and delay-tolerant DML tasks, this paper introduces multi-renewable energy in the geo-distributed continuum to address this issue, the spatiotemporal complementarity maximizes the renewable energy utilization and compensates for time-dependent energy differences with geographic advantages. Additionally, considering the dynamic differences in carbon intensity and electricity prices across distributed DCs in the continuum, we propose an energy and carbon-aware algorithm called ECMR for scheduling heterogeneous virtual machine creation tasks of DML among multi-clouds in different time zones. It is demonstrated that compared with the baseline methods, the ECMR significantly reduces the total power consumption, energy cost, and carbon emission of data centers while maintaining an acceptable service quality. The utilization of renewable energy in data centers has been significantly improved to 90.8% by flexibly leveraging the spatiotemporal complementarity of multi-renewable energy. Compared with existing competing algorithms, the proposed method exhibits significant improvements with an achieved average response time of 12.6 ms, and a task failure rate of 1.25%.
30
Vinod Kumar Vavilapalli, Arun C. Murthy, Chris Douglas et al. · 2013 · 1.8K citations
Large-scale cluster management at Google with Borg
Abhishek Verma, Luis Pedrosa, Madhukar Korupolu et al. · 2015 · 1.3K citations · Full text
Learning scheduling algorithms for data processing clusters
Hongzi Mao, Malte Schwarzkopf, Shaileshh Bojja Venkatakrishnan et al. · 2019 · 663 citations · Full text