MISO

TLDR

Multistore systems combine distinct data stores such as HDFS and RDBMS, but existing query‑processing methods suffer from high data‑movement costs, and tuning the physical design to decide where data resides can reduce movement and improve performance. The study proposes the first method to tune a multistore system’s physical design by deciding where to place data. MISO is an adaptive, lightweight, online tuning method that uses opportunistic views—by‑products of query processing—to guide data placement. MISO significantly boosts ad‑hoc big‑data query performance with minimal overhead by exploiting each store’s characteristics.

Abstract

Multistore systems utilize multiple distinct data stores such as Hadoop's HDFS and an RDBMS for query processing by allowing a query to access data and computation in both stores. Current approaches to multistore query processing fail to achieve the full potential benefits of utilizing both systems due to the high cost of data movement and loading between the stores. Tuning the physical design of a multistore, i.e., deciding what data resides in which store, can reduce the amount of data movement during query processing, which is crucial for good multistore performance. In this work, we provide what we believe to be the first method to tune the physical design of a multistore system, by focusing on which store to place data. Our method, called MISO for MultISstore Online tuning, is adaptive, lightweight, and works in an online fashion utilizing only the by-products of query processing, which we term as opportunistic views. We show that MISO significantly improves the performance of ad-hoc big data query processing by leveraging the specific characteristics of the individual stores while incurring little additional overhead on the stores.