Large-scale data visualization using parallel data streaming

Abstract

We present an architectural approach based on parallel data streaming to enable visualizations on a parallel cluster. Our approach requires less memory than other visualizations while achieving high code reuse. We implemented our architecture within the Visualization Toolkit (VTK). It includes specific additions to support message passing interfaces (MPIs); memory limit-based streaming of both implicit and explicit topologies; translation of streaming requests between topologies; and passing data and pipeline control between shared, distributed, and mixed memory configurations. The architecture directly supports both sort-first and sort-last parallel rendering.