Architectural support for operating system-driven CMP cache management

Abstract

The role of the operating system (OS) in managing shared resources such as CPU time, memory, peripherals, and even energy is well motivated and understood [23]. Unfortu-nately, one key resource|lower-level shared cache in chip multi-processors|is commonly managed purely in hardware by rudimentary replacement policies such as least-recently-used (LRU). The rigid nature of the hardware cache manage-ment policy poses a serious problem since there is no single best cache management policy across all sharing scenarios. For example, the cache management policy for a scenario where applications from a single organization are running under \\best eort " performance expectation is likely to be dierent from the policy for a scenario where applications from competing business entities (say, at a third party data center) are running under a minimum service level expecta-tion. When it comes to managing shared caches, there is an inherent tension between \nexibility and performance. On one hand, managing the shared cache in the OS oers im-mense policy \nexibility since it may be implemented in soft-ware. Unfortunately, it is prohibitively expensive in terms of performance for the OS to be involved in managing tempo-rally ne-grain events such as cache allocation. On the other hand, sophisticated hardware-only cache management tech-niques to achieve fair sharing or throughput maximization have been proposed. But they oer no policy \nexibility. This paper addresses this problem by designing architec-tural support for OS to eciently manage shared caches with a wide variety of policies. Our scheme consists of a hard-ware cache quota management mechanism, an OS interface and a set of OS level quota orchestration policies. The hard-ware mechanism guarantees that OS-specied quotas are en-forced in shared caches, thus eliminating the need for (and the performance penalty of) temporally ne-grained OS in-tervention. The OS retains policy \nexibility since it can tune the quotas during regularly scheduled OS interventions. We demonstrate that our scheme can support a wide range of policies including policies that provide (a) passive per-