Inside the Social Network's (Datacenter) Network

TLDR

Large cloud providers build massive datacenters, prompting research into efficient network fabrics, yet operators rarely disclose application requirements and existing workload data are limited to a single operator. This study reports on the network traffic observed in some of Facebook's datacenters. Facebook’s datacenter traffic shows higher locality, stability, and predictability than literature reports, challenging assumptions about traffic patterns and informing network architecture, traffic engineering, and switch design.

Abstract

Large cloud service providers have invested in increasingly larger datacenters to house the computing infrastructure required to support their services. Accordingly, researchers and industry practitioners alike have focused a great deal of effort designing network fabrics to efficiently interconnect and manage the traffic within these datacenters in performant yet efficient fashions. Unfortunately, datacenter operators are generally reticent to share the actual requirements of their applications, making it challenging to evaluate the practicality of any particular design. Moreover, the limited large-scale workload information available in the literature has, for better or worse, heretofore largely been provided by a single datacenter operator whose use cases may not be widespread. In this work, we report upon the network traffic observed in some of Facebook's datacenters. While Facebook operates a number of traditional datacenter services like Hadoop, its core Web service and supporting cache infrastructure exhibit a number of behaviors that contrast with those reported in the literature. We report on the contrasting locality, stability, and predictability of network traffic in Facebook's datacenters, and comment on their implications for network architecture, traffic engineering, and switch design.