The ATLAS Distributed Data Management project: Past and Future

Abstract

<!--HTML-->The ATLAS collaboration has recorded almost 5PB of RAW data since the LHC started running at the end of 2009. Together with experimental data generated from RAW and complimentary simulation data, and accounting for data replicas on the grid, a total of 74TB is currently stored in the Worldwide LHC Computing Grid by ATLAS. All of this data is managed by the ATLAS Distributed Data Management system, called Don Quixote 2 (DQ2).\nThe DQ2 system has over time rapidly evolved to assist the ATLAS collaboration management to properly manage the data, as well as provide an effective interface allowing physicists easy access to this data. Numerous\nnew requirements and operational experience of ATLAS' use cases have necessitated the need for a next generation data management system, called Rucio, which will re-engineer the current system to cover new high-level use cases and workflows such as the management of data for physics groups.\nIn this talk, we will describe the state of the current of DQ2, and present an overview of the upcoming Rucio system, covering it's architecture, new innovative features, and preliminary benchmarks.