Speculative defragmentation - a technique to improve the communication software efficiency for Gigabit Ethernet

Abstract

Cluster platforms offer good computational performance, but they still cannot utilize the potential of Gbit/s communication technology. While the speed of the Ethernet has grown to 1 Gbit/s, the functionality and the architectural support in the network interfaces has remained the same for more than a decade, so that the memory system becomes a limiting factor. To sustain the raw network speed in applications, a "zero-copy" network interface architecture would be required, but, for all widely used stacks, a last copy is required for the (de)fragmentation of the transferred network packets, since Ethernet packets are smaller than a page size. Correctly defragmenting packets of various communication protocols in hardware is an extremely complex task. We therefore consider a speculative defragmentation technique that can eliminate the last defragmenting copy operation in zero-copy TCP/IP stacks on existing hardware. The payload of fragmented packets is separated from the headers and stored in a memory page that can be mapped directly to its final destination in user memory. To evaluate our ideas, we integrated a network interface driver with speculative defragmentation into an existing protocol stack and added well-known page remapping and fast buffer strategies. Measurements indicate that we can improve the performance for a Gigabit Ethernet over a standard Linux 2.2 TCP/IP stack by a factor of 1.5-2 for uninterrupted burst transfers. Furthermore, our study demonstrates good speculation success rates for a database and a scientific application code on a cluster of PCs.