A Flattened Metadata Service for Distributed File Systems

Abstract

Key-Value stores provide scalable metadata service for distributed file systems. However, the metadata's organization itself, which is organized using a directory tree structure, does not fit the key-value access pattern, thereby limiting the performance. To address this issues, we propose a distributed file system with a flattened and fine-grained division metadata service, LocoMeta, to bridge the performance gap between file system metadata and key-value stores. LocoMeta is designed to bridge the gap between file metadata to key-value store with two techniques. First, LocoMeta flattens the directory content and structure, which organizes file and directory index nodes in a flat space while reversely indexing the directory entries. Second, it exploits a fine-grained division method to improve the key-value access performance. Evaluations show that LocoMeta with eight nodes boosts the metadata throughput by five times, which approaches 93 percent throughput of a single-node key-value store, compared to 18 percent in the state-of-the-art IndexFS.