2019 · 12 citations · 39 references
ComputingComputer ScienceComputer SystemsCloud ArchitectureCloud Performance OptimizationQueuing TheoryCompute Resource ManagementCpu SchedulingLatency OptimizationQueueing TheoryResource SchedulingTask SchedulingBlockingHardware SchedulingHead-of-line BlockingLatency ReductionMicroservice SchedulingTail LatencyTail Latency ReductionWork Stealing
Managing high-percentile tail latencies is key to designing user-facing cloud microservices. A main contributor to end-to-end tail latency is queuing, wherein nominal tasks are enqueued behind rare, long ones, due to head-of-line blocking. In this paper, we propose Express-Lane SMT (ESMT), which extends the hardware scheduling of a simultaneously multithreaded (SMT) core to provide an "express-lane" execution context for short tasks, protecting them from queuing behind rare, long ones. As tasks reach predefined service cutoffs, ESMT preempts and migrates them to the subsequent queue to be serviced by the next SMT execution lane, thereby preventing Head-of-Line (HoL) blocking. We further propose an enhanced variant of ESMT that allows execution lanes to work-steal from each other to maximize utilization. Our evaluation shows that ESMT with work stealing reduces tail latency over a conventional SMT core by an average of 56% and 67% under moderate (40%) and high (70%) system loads, respectively.
39
Nathan Binkert, Bradford M. Beckmann, Gabriel Black et al. · ACM SIGARCH Computer Architecture News · 2011 · 4.9K citations
Luiz André Barroso · Communications of the ACM · 2013 · 1.7K citations
Michael Ferdman, Almutaz Adileh, Onur Kocberber et al. · 2012 · 814 citations
Dean M. Tullsen, Susan J. Eggers, Joel Emer et al. · 1996 · 782 citations
Rajesh Nishtala, Hans Fugal, Steven Grimm et al. · Networked Systems Design and Implementation · 2013 · 597 citations