FID: A Faster Image Distribution System for Docker Platform

Abstract

Docker has been widely adopted in enterprise-level container environment. As an important part of Docker-based container ecosystem, Docker Registry provides the service of storing, distributing and managing Docker images, which is crucial to run Docker containers. In large-scale container platforms, deploying applications is prone to overburdening Docker Registry with flooding network traffic, and this situation may even cause failures of image services. In this paper, we present a new P2P-based large-scale image distribution system called FID (Faster Image Distribution), which is able to accelerate the speed of distributing Docker images by taking full advantage of the bandwidth of not only Docker Registry but also other nodes in the cluster. We implemented and validated FID on the enterprise-level cluster. The experiment results show that, compared to the native image distribution method, FID reduces at least 97% of network traffic for Docker Registry. Furthermore, it reduces 83.50% of distribution time on average when distributing images among 200 nodes, and particularly, reduces up to 91.35% of distribution time for the 500M image of Hadoop.