Performance Issues in Incremental Warehouse Maintenance

Abstract

A well-known challenge in data warehousing is the efficient incremental maintenance of warehouse data in the presence of source data updates. In this paper, we identify several critical data representation and algorithmic choices that must be made when developing the machinery of an incrementally maintained data warehouse. For each decision area, we identify various alternatives and evaluate them through extensive experiments. We show that picking the right alternative can lead to dramatic performance gains, and we propose guidelines for making the right decisions under different scenarios. All of the issues addressed in this paper arose in our development of WHIPS, a prototype data warehousing system supporting incremental maintenance. 1 Introduction Data warehousing systems integrate and store data from remote sources as materialized views in the warehouse [LW95, CD97]. When source data changes, warehouse views need to be maintained so that they remain consistent with the so...