ACCL: FPGA-Accelerated Collectives over 100 Gbps TCP-IP

Abstract

Collective operations such as scatter, gather, reduce, etc are utilized broadly to implement distributed HPC applications and are the target of extensive optimization in all MPI implementations as well as dedicated collective libraries by accelerator vendors (e.g. NCCL and RCCL by NVidia and AMD respectively). We present ACCL, an open-source FPGA-accelerated collectives library designed to serve applications running primarily in Xilinx FPGAs. Compared to previous collective communication solutions for FPGA, ACCL is flexible and extensible, easily portable, and fast. We evaluate ACCL up to 8 nodes and demonstrate that ACCL outperforms OpenMPI over 100 Gbps TCP-IP for large messages.