Generating representative Web workloads for network and server performance evaluation

TLDR

Workload generation helps understand how servers and networks respond to load variation, aiding management and capacity planning. The paper reviews essential elements for generating representative Web workloads and applies observations of Web server usage to create a realistic workload generation tool that mimics real users accessing a server. The Surge tool generates URL references that match empirical distributions of file size, request size, file popularity, embedded references, temporal locality, and user idle periods, and it addresses technical challenges to satisfy these constraints with accurate solutions. Surge exercises servers in a manner significantly different from other Web server benchmarks.

Abstract

One role for workload generation is as a means for understanding how servers and networks respond to variation in load. This enables management and capacity planning based on current and projected usage. This paper applies a number of observations of Web server usage to create a realistic Web workload generation tool which mimics a set of real users accessing a server. The tool, called Surge (Scalable URL Reference Generator) generates references matching empirical measurements of 1) server file size distribution; 2) request size distribution; 3) relative file popularity; 4) embedded file references; 5) temporal locality of reference; and 6) idle periods of individual users. This paper reviews the essential elements required in the generation of a representative Web workload. It also addresses the technical challenges to satisfying this large set of simultaneous constraints on the properties of the reference stream, the solutions we adopted, and their associated accuracy. Finally, we present evidence that Surge exercises servers in a manner significantly different from other Web server benchmarks.