NetCache

TLDR

NetCache is a key‑value store architecture that uses programmable switches to handle hot‑item queries and balance load across storage nodes. NetCache’s mechanism is a packet‑processing pipeline on programmable switch ASICs that detects, indexes, caches, and serves hot key‑value items, enabling a single switch to process over 2 billion queries per second for 64 K items while using only a small fraction of hardware resources. NetCache achieves line‑rate performance, delivering over 2 billion queries per second with high throughput and low latency, while guaranteeing cache coherence, and improving throughput 3–10× and reducing latency by up to 50% compared to conventional in‑memory key‑value stores.

Abstract

We present NetCache, a new key-value store architecture that leverages the power and flexibility of new-generation programmable switches to handle queries on hot items and balance the load across storage nodes. NetCache provides high aggregate throughput and low latency even under highly-skewed and rapidly-changing workloads. The core of NetCache is a packet-processing pipeline that exploits the capabilities of modern programmable switch ASICs to efficiently detect, index, cache and serve hot key-value items in the switch data plane. Additionally, our solution guarantees cache coherence with minimal overhead. We implement a NetCache prototype on Barefoot Tofino switches and commodity servers and demonstrate that a single switch can process 2+ billion queries per second for 64K items with 16-byte keys and 128-byte values, while only consuming a small portion of its hardware resources. To the best of our knowledge, this is the first time that a sophisticated application-level functionality, such as in-network caching, has been shown to run at line rate on programmable switches. Furthermore, we show that NetCache improves the throughput by 3-10x and reduces the latency of up to 40% of queries by 50%, for high-performance, in-memory key-value stores.