Exploring distributed hash tables in HighEnd computing

Abstract

Over the last decade, storage systems have experienced a 10-fold increase between their capacity and bandwidth. This gap is predicted to grow faster with exponentially growing concurrency levels, with future exascales delivering millions of nodes and billions of threads of execution. A critical component of future file systems for high-end computing is metadata management. This extended abstract presents ZHT, a zero-hop distributed hash-table, which has been tuned for the specific requirements of high-end computing. The primary goal of ZHT is excellent availability, fault tolerance, high throughput, and low latencies.