CacheLib is a general purpose caching engine that facilitates the easy development, scaling and maintenance of high performing caches. CacheLib was designed by drawing on experiences across a range of caching use cases at Facebook. From its first deployment in 2017, CacheLib now powers over 70 services at Facebook including Facebook’s CDN, social-graph cache, application look-aside cache, and block-storage system.
Large scale web service providers rely on caching to deliver a great user experience. While web applications commonly leverage remote look-aside caches, several other applications need a local in-process cache because, either they can not tolerate the RPC overhead of a common remote cache, or require strict cache consistency, or need domain specific cache features. These application teams often build and maintain application specific, highly specialized, local in-process caches. Without the ability to share improvements among each other, this approach leads to each team solving the hard problems of scaling cache performance in isolation.
CacheLib is a thread-safe, scalable, C++ library that provides the core caching functionality. It enables services to customize and scale highly concurrent caches easily, and to leverage the improvements across different caching systems by using a simple, expressive, thread-safe API. See docs for more info.
- Efficient implementations of caching indexes, eviction policies
- Support for seamless hybrid caching (caches composed of DRAM and Flash) to achieve high hit ratios while caching large working sets. Relevant for content delivery cache use cases.
- Optimizations for high throughput, low memory, and low CPU usage for a broad range of workloads.
- Native implementations of arrays and hashmaps that can be cached and mutated efficiently without incurring any serialization overhead.
- Ability to perform application binary restarts and retain the state of the cache
CacheLib Caching Engine: Design and Experiences at Scale was published at the USENIX OSDI` 20 conference. The publication can be accessed here and the video of the presentation is below.
Kangaroo: Caching Billions of Tiny Objects on Flash was published at the SOSP 2021 conference, and won the best paper award! The publication can be accessed here and the video of the presentation is below.