Cost Models DO Matter: Providing Cost Information for Diverse Data Sources in a Federated System

TLDR

Federated systems must optimize cross‑source queries without embedding source knowledge, and Garlic exemplifies an extensible system handling diverse data sources. The paper proposes a framework enabling wrappers to supply cost and cardinality data for query optimization in Garlic. Wrappers are attached to Garlic via a wrapper interface that participates in query planning, indicating which query fragments a source can execute and the associated cost. The framework simplifies wrapper cost reporting, minimally alters a bottom‑up optimizer, and, as the first such system, delivers accurate cost estimates that enable Garlic to select effective plans for complex cross‑source queries.

Abstract

An important issue for federated systems of diverse data sources is how to optimize cross-source queries, without building knowledge of individual sources into the optimizer. Garlic is a federated system with an emphasis on extensibility and diverse sources. To achieve these goals, data sources are attached to Garlic by means of a wrapper. Wrappers participate in query planning, telling Garlic what parts of a query a data source can do and how much it will cost. This paper describes a framework through which wrappers provide the necessary cost and cardinality information for optimization, and the facilities Garlic provides to make this task easier. Our framework makes it easy for wrappers to provide cost information, requires few changes to a conventional bottomup optimizer and is easily extensible to a broad range of sources. We believe that our framework for costing is the first to allow accurate cost estimates for diverse sources within the context of a traditional cost-based optimizer. We demonstrate the importance of cost information in choosing good plans, the flexibility of our framework, the accuracy it allows, and finally, that it works – the optimizer is able to choose good plans even for complex cross-source queries.