Policy-Based Information Lifecycle Management in a Large-Scale File System

Abstract

Policy-based file lifecycle management is important for balancing storage utilization and for regulatory conformance. It poses two important challenges, the need for simple yet effective policy design and an implementation that scales to billions of files. This paper describes the design and an innovative implementation technique of policy-based lifecycle management in a prototype built as a part of IBM's new SAN file system. The policy specification leverages a key abstraction in the file system called storage pools and its ability to support location independence for files. The policy implementation uses an innovative new technique that combines concurrent policy execution and a policy decisions cache, to enable scaling to billions of files under normal usage patterns.