Optimizing strided remote memory access operations on the Quadrics QsNetII network interconnect

Abstract

This paper describes and evaluates protocols for optimizing strided noncontiguous communication on the Quadrics QsNetII high-performance network interconnect. Most of previous related studies focused primarily on NIC-based or host-based protocols. This paper discusses merits for using both approaches and tries to determine types and data sizes in the communication operations for which these protocols should be used. We focus on the Quadrics QsNetll network, which offers powerful communication processors on the network interface card (NIC) and practical and flexible opportunities for exploiting them in context of the user. Furthermore, the paper focuses on noncontiguous data remote memory access (RMA) transfers and performs the evaluation in context of standalone communication and application microbenchmarks. In comparison to the vendor provided noncontiguous interfaces, proposed approach achieved significant performance improvement in context of microbenchmarks as well as application kernels; dense matrix multiplication, and the Co-Array Fortran version of the NAS BT parallel benchmark. For example, for NAS BT Class B, 54 % improvement in overall communication time and a 42% improvement in matrix multiplication was achieved for 64 processes.