Extreme-scale scripting: Opportunities for large task-parallel applications on petascale computers

Abstract

Abstract. Parallel scripting is a loosely-coupled programming model in which applications are composed of highly parallel scripts of program invocations that process and exchange data via files. We characterize here the applications that can benefit from parallel scripting on petascale-class machines, describe the mechanisms that make this feasible on such systems, and present results achieved with parallel scripts on currently available petascale computers. 1. The parallel scripting paradigm John Ousterhout describes scripting as higher-level programming for the 21st Century [3]. Scripting has revolutionized application development on the desktop and server, accelerating and simplifying programming by focusing on the composition of programs to form more powerful applications. Understanding how to scale scripting to 21st century computers should be among the priorities for researchers of next generation parallel programming models. Might scripting not provide the same benefits for extreme-scale computers? We believe that the answer to this question is yes. We introduce the concept and implementation of parallel scripting, and describe what becomes possible when simple scripts