ComputingComputer EngineeringComputer ScienceHeterogeneous ComputingParallel ComputingVirtualizationCachingCaching StrategiesCompute Resource ManagementCpu SchedulingMemory HierarchyMemory Hierarchy DesignCache ArchitectureCache LocalityCache ManagementMemory Access LatencyMemory Access LocalityPriority AssignmentSpatial LocalityVirtual Machines
Cache hierarchies have traditionally been designed for single‑application use, but emerging multi‑threaded and multi‑core platforms generate heterogeneous memory‑access streams that conventional cache management treats uniformly, leading to inefficient space utilization and poor performance. This paper introduces CQoS, a cache management framework that recognizes memory‑access heterogeneity, applies QoS concepts, and assigns priorities based on latency sensitivity, locality, and application performance requirements. CQoS implements priority classification, assignment, and enforcement through user‑, developer‑, compiler‑, and flow‑based options, selective cache allocation, static/dynamic set partitioning, and heterogeneous cache regions, and its architectural design and implementation complexity are evaluated in a cache simulator on CMP platforms running network‑intensive server workloads. Simulation results demonstrate that CQoS improves shared cache efficiency and overall system performance, supporting its adoption in future multi‑threaded/multi‑core platforms.
Cache hierarchies have been traditionally designed for usage by a single application, thread or core. As multi-threaded (MT) and multi-core (CMP) platform architectures emerge and their workloads range from single-threaded and multithreaded applications to complex virtual machines (VMs), a shared cache resource will be consumed by these different entities generating heterogeneous memory access streams exhibiting different locality properties and varying memory sensitivity. As a result, conventional cache management approaches that treat all memory accesses equally are bound to result in inefficient space utilization and poor performance even for applications with good locality properties. To address this problem, this paper presents a new cache management framework (CQoS) that (1) recognizes the heterogeneity in memory access streams, (2) introduces the notion of QoS to handle the varying degrees of locality and latency sensitivity and (3) assigns and enforces priorities to streams based on latency sensitivity, locality degree and application performance needs. To achieve this, we propose CQoS options for priority classification, priority assignment and priority enforcement. We briefly describe CQoS priority classification and assignment options -- ranging from user-driven and developer-driven to compiler-detected and flow-based approaches. Our focus in this paper is on CQoS mechanisms for priority enforcement -- these include (1) selective cache allocation, (2) static/dynamic set partitioning and (3) heterogeneous cache regions. We discuss the architectural design and implementation complexity of these CQoS options. To evaluate the performance trade-offs for these options, we have modeled these CQoS options in a cache simulator and evaluated their performance in CMP platforms running network-intensive server workloads. Our simulation results show the effectiveness of our proposed options and make the case for CQoS in future multi-threaded/multi-core platforms since it improves shared cache efficiency and increases overall system performance as a result.
28
Wm. A. Wulf, Sally A. McKee · ACM SIGARCH Computer Architecture News · 1995 · 1.5K citations
Dean M. Tullsen, Susan J. Eggers, Henry M. Levy · 1995 · 1.4K citations · Full text
Hitting the Memory Wall: Implications of the Obvious
Wm. A. Wulf, Sally A. McKee · 1995 · 1.4K citations