Data Integration Needs an Industrial Revolution

Abstract

The data resources in a large enterprise typically exist as many separate islands of data. Each is maintained by a distinct community for its purposes, and is largely unusable by others. It is common to see whole data archipelagos comprised of thousands of separate resources [Ston00]. We would, of course, prefer to see one single integrated data resource usable by all. This is the “grand vision” of data integration: discovery of and access to all data, with multiple sources properly combined, delivered in a form that each consumer can interpret. Decentralized organizations such as the US Air Force might accept for now a slightly less ambitious dream – the ability to establish a connection between islands, a way to obtain any desired information from any other source.