The Quadrics network (QsNet): high-performance clustering technology

TLDR

The Quadrics interconnection network (QsNet) introduces a global virtual address space and fault‑tolerant link protocols, and its features anticipate those of InfiniBand. We present an initial performance evaluation of QsNet. QsNet extends node operating systems with a network OS and specialized NIC hardware, and the authors benchmarked its hardware and software features on an Intel‑based Linux cluster. Initial analysis shows QsNet achieves user‑level latency under 2 µs and bandwidth over 300 MB/s.

Abstract

The Quadrics interconnection network (QsNet) contributes two novel innovations to the field of high-performance interconnects: (I) integration of the virtual-address spaces of individual nodes into a single, global, virtual-address space and (2) network fault tolerance via link-level and end-to-end protocols that can detect faults and automatically re-transmit packets. QsNet achieves these feats by extending the native operating system in the nodes with a network operating system and specialized hardware support in the network interface. As these and other important features of QsNet can be found in the InfiniBand specification, QsNet can be viewed as a precursor to InfiniBand. In this paper, we present an initial performance evaluation of QsNet. We first describe the main hardware and software features of QsNet, followed by the results of benchmarks that we ran on our experimental, Intel-based, Linux cluster built around QsNet. Our initial analysis indicates that QsNet performs remarkably well, e.g., user-level latency under 2 /spl mu/s and bandwidth over 300 MB/s.