Large-scale adaptive mesh simulations through non-volatile byte-addressable memory

Abstract

Octree-based mesh adaptation has enabled simulations of complex physical phenomena. Existing meshing algorithms were proposed with the assumption that computer memory is volatile. Consequently, for failure recovery, the in-core algorithms need to save memory states as snapshots with slow file I/Os. The out-of-core algorithms store octants on disks for persistence. However, neither of them was designed to leverage unique characteristics of non-volatile byte-addressable memory (NVBM). In this paper, we propose a novel data structure Persistent Merged octree (PM-octree) for both meshing and in-memory storage of persistent octrees using NVBM. It is a multi-version data structure and can recover from failures using its earlier persistent version stored in NVBM. In addition, we design a feature-directed sampling approach to help dynamically transform the PM-octree layout for reducing NVBM-induced memory write latency. PM-octree has been successfully integrated with Gerris software for simulation of fluid dynamics. Our experimental results with real-world scientific workloads show that PM-octree scales up to 1.1 billion mesh elements with 1000 processors on the Titan supercomputer.