Requirements for DataParallel Programming Environments

Abstract

Over the past decade, research in programming systems to support scalable parallel computation has sought ways to provide an efficient machine-independent programming model. Initial efforts concentrated on automatic detection of parallelism using extensions to compiler technology developed for automatic vectorization. Many advanced techniques were tried. However, after over a half-decade of research, most investigators were ready to admit that fully automatic techniques would be insufficient by themselves to support general parallel programming, even in the limited domain of scientific computation. In other words, in an effective parallel programming system, the programmer would have to provide additional information to help the system parallelize applications. This realization led the research community to consider extensions to existing programming languages, such as Fortran and C, that could be used to help specify parallelism. An important strategy for exploiting scalable parallelism is the use of data parallelism, in which the problem domain is subdivided into regions and each region is mapped onto a different processor. These factors have led to a widespread interest in data-parallel languages such as Fortran D, High Performance Fortran (HPF), and DataParallel C as a means of writing portable parallel software. To help the programmer make good design decisions, the programming system should include mechanisms that explain the behavior of object code in terms of the source program from which it was compiled. For sequential programs, the standard "symbolic debugger," supporting single-step execution of the program source rather than the object program, provides such a facility. A more recent example is the "interactive vectorizer." The goal of this paper is to convey an understanding of the tools and strategies that will be needed to adequately support efficient, machine-independent, data-parallel programming.