A fine-grain parallel architecture based on barrier synchronization

Abstract

Although barrier synchronization has long been considered a useful construct for parallel programming, it has generally been either layered on top of a communication system or used as a completely independent mechanism. Instead, we propose that all communication be made a side-effect of barrier synchronization. This is done by extending the barrier synchronization unit to collect a datum from each processor, compute an aggregate function, and return the corresponding result to each processor. This paper describes a scalable prototype implementation of PAPERS (Purdue's Adapter for Parallel Execution and Rapid Synchronization). Despite the fact that the prototype is implemented as very simple TTL hardware connecting conventional workstations, measured performance on fine-grain parallel communication operations is far superior to that obtained using conventional workstation networks. It is comparable to the performance of commercially available supercomputers.