The end of an architectural era: (it's time for a complete rewrite)

TLDR

Relational DBMSs have long been the default, but emerging specialized engines are set to dominate data‑warehouse, stream, text, and scientific markets, leaving traditional RDBMSs largely confined to OLTP and hybrid workloads. This study demonstrates that current RDBMSs can be outperformed by nearly two orders of magnitude in OLTP and urges vendors and researchers to abandon legacy architectures in favor of new, purpose‑built systems. We benchmarked MIT’s H‑Store OLTP prototype against a popular RDBMS on the TPC‑C transaction benchmark to generate the evidence. Experimental results show that specialized engines beat legacy RDBMSs by 1–2 orders of magnitude across multiple domains, confirming that the current code base excels at nothing and should be retired in favor of fresh, specialized engines.

Abstract

In previous papers [SC05, SBC+07], some of us predicted the end of size fits all as a commercial relational DBMS paradigm. These papers presented reasons and experimental evidence that showed that the major RDBMS vendors can be outperformed by 1--2 orders of magnitude by specialized engines in the data warehouse, stream processing, text, and scientific database markets. Assuming that specialized engines dominate these markets over time, the current relational DBMS code lines will be left with the business data processing (OLTP) market and hybrid markets where more than one kind of capability is required. In this paper we show that current RDBMSs can be beaten by nearly two orders of magnitude in the OLTP market as well. The experimental evidence comes comparing a new OLTP prototype, H-Store, which we have built at M.I.T. to a popular RDBMS on the standard transactional benchmark, TPC-C. We conclude that the current RDBMS code lines, while attempting to be a size fits all solution, in fact, excel at nothing. Hence, they are 25 year old legacy code lines that should be retired in favor of a collection of from scratch specialized engines. The DBMS vendors (and the research community) should start with a clean sheet of paper and design systems for tomorrow's requirements, not continue to push code lines and architectures designed for yesterday's needs.