Beyond macrobenchmarks

TLDR

Graph database requirements and specifications remain poorly understood, causing wide variation in supported features and performance. The study aims to comprehensively evaluate existing graph database systems and introduce a microbenchmarking framework that offers deeper performance insights than macro‑benchmarks. The framework incorporates the largest set of queries and operators to date, evaluating systems on synthetic and real datasets from diverse domains at unprecedented scales. The framework is released as an open‑source suite that can be readily extended to new datasets, systems, and queries.

Abstract

Despite the increasing interest in graph databases their requirements and specifications are not yet fully understood by everyone, leading to a great deal of variation in the supported functionalities and the achieved performances. In this work, we provide a comprehensive study of the existing graph database systems. We introduce a novel microbenchmarking framework that provides insights on their performance that go beyond what macro-benchmarks can offer. The framework includes the largest set of queries and operators so far considered. The graph database systems are evaluated on synthetic and real data, from different domains, and at scales much larger than any previous work. The framework is materialized as an open-source suite and is easily extended to new datasets, systems, and queries1 .