2003 · 15 citations · 23 references
ComputingComputer ScienceComputer SystemsData ManagementDistributed SystemsStorage SystemsData Storage SystemsDatabase ManagementDistributed StorageFile SystemsStorage ManagementStorage TieringData Management PlatformsData Storage ArchitecturesDistributed Storage ArchitecturesHierarchical Storage ManagementHigh PerformanceHigh-performance Computing SystemsThree-tier Architecture
With the increasing number of scientific applications manipulating huge amounts of data, effective data management is an increasingly important problem. Unfortunately, so far the solutions to this data management problem either require deep understanding of specific storage architectures and file layouts (as in high-performance file systems) or produce unsatisfactory I/O performance in exchange for ease-of-use and portability (as in relational DBMSs). In this paper we present a new environment which is built around an active meta-data management system (MDMS). The key components of our three-tiered architecture are user application, the MDMS, and a hierarchical storage system (HSS). Our environment overcomes the performance problems of pure database-oriented solutions, while maintaining their advantages in terms of ease-of-use and portability. The high levels of performance are achieved by the MDMS, with the aid of user-specified directives. Our environment supports a simple, easy-to-use yet powerful user interface, leaving the task of choosing appropriate I/O techniques to the MDMS. We discuss the importance of an active MDMS and show how the three components, namely application, the MDMS, and the HSS, fit together. We also report performance numbers from our initial implementation and illustrate that significant improvements are made possible without undue programming effort.
23
The SDSC storage resource broker
Chaitanya Baru, Reagan Moore, Arcot Rajasekar et al. · 2010 · 557 citations
Data sieving and collective I/O in ROMIO
Rajeev Thakur, William Gropp, Ewing Lusk · 1999 · 498 citations · Full text
Input/output characteristics of scalable parallel applications
Phyllis E. Crandall, Ruth Aydt, Andrew A. Chien et al. · 1995 · 178 citations · Full text