CASTOR: A Distributed Storage Resource Facility for High Performance Data Processing at CERN

Abstract

Mass storage systems at CERN have evolved over time to meet growing requirements, in terms of both scalability and fault resiliency. The CERN advanced storage system (CASTOR) and its new disk cache management layer (CASTOR2) have been developed to meet the challenges raised by the experiments using the new accelerator that CERN is building: the large hadron collider (LHC) [4]. This system must be able to cope with hundreds of millions of files, tens of petabytes of storage and handle a constant throughput of several gigabytes per second. In this paper, we detail CASTOR's architecture and implementation and present some operational aspects. We finally list the performance levels achieved by the current version both in a production environment and during internal tests.