2020 · 10 citations · 20 references
ComputingComputer EngineeringComputer ScienceComputer NetworksHeterogeneous ComputingParallel ComputingHeterogeneous System ArchitectureResource SchedulingCommunication EfficiencyData Center NetworkFlow SchedulingHeterogeneous NetworksHeterogeneous Resource ManagementHeterogeneous SchedulingNetwork-aware SchedulingScheduler Design
Leveraging application-level requirements expressed in Coflows has been shown to improve application-level communication efficiency. However, most existing works assume all application traffic is serviced by one monolithic network. This over-simplified assumption is no longer sufficient in a modern, evolving data center which operates on multiple generations of network fabrics, an architecture that we define as Heterogeneous Parallel Networks (HPNs). In this paper, we present the first scheduler, called Weaver, that addresses the Coflow management problem in HPNs. To exploit HPNs fully, achieving high communication efficiency for applications is crucial, yet it is also challenging because of the complex traffic patterns in applications and the heterogeneous bandwidth distribution in HPNs. Weaver addresses these challenges at two levels. At the microscopic level, for each application, Weaver leverages an efficient algorithm to exploit the distributed bandwidth in HPNs, which we proved to be within a constant factor of the optimal. At the macroscopic level involving multiple applications, Weaver can adopt a range of application traffic scheduling policies as desired by the system operator. Under realistic traffic, Weaver enables HPNs to service Coflows as efficiently as a monolithic network with equivalent aggregated capacity.
20
Nick McKeown, Tom Anderson, Hari Balakrishnan et al. · ACM SIGCOMM Computer Communication Review · 2008 · 8.4K citations
Albert Greenberg, James R. Hamilton, Navendu Jain et al. · 2009 · 2.1K citations
A scalable, commodity data center network architecture
Mohammad Al-Fares, Alexander Loukissas, Amin Vahdat · 2008 · 1.7K citations · Full text
Hedera: dynamic flow scheduling for data center networks
Mohammad Al-Fares, Sivasankar Radhakrishnan, Barath Raghavan et al. · 2010 · 1.5K citations