Foyer
Hybrid in-memory and disk cache in Rust
Install / Use
/learn @foyer-rs/FoyerREADME
foyer
foyer aims to be an efficient and user-friendly hybrid cache lib in Rust.
foyer draws inspiration from Facebook/CacheLib, a highly-regarded hybrid cache library written in C++, and ben-manes/caffeine, a popular Java caching library, among other projects.
However, foyer is more than just a rewrite in Rust effort; it introduces a variety of new and optimized features.
For more details, please visit foyer's website: https://foyer-rs.github.io/foyer 🥰
Website | Tutorial | API Docs | Crate
Features
- Hybrid Cache: Seamlessly integrates both in-memory and disk cache for optimal performance and flexibility.
- Plug-and-Play Algorithms: Empowers users with easily replaceable caching algorithms, ensuring adaptability to diverse use cases.
- Fearless Concurrency: Built to handle high concurrency with robust thread-safe mechanisms, guaranteeing reliable performance under heavy loads.
- Zero-Copy In-Memory Cache Abstraction: Leveraging Rust's robust type system, the in-memory cache in foyer achieves a better performance with zero-copy abstraction.
- User-Friendly Interface: Offers a simple and intuitive API, making cache integration effortless and accessible for developers of all levels.
- Out-of-the-Box Observability: Integrate popular observation systems such as Prometheus, Grafana, Opentelemetry, and Jaeger in just ONE line.
Projects Using foyer
Feel free to open a PR and add your projects here:
- RisingWave: SQL stream processing, analytics, and management.
- Chroma: Embedding database for LLM apps.
- SlateDB: A cloud native embedded storage engine built on object storage.
- ZeroFS: File systems and block devices on S3 storage.
- Percas: A distributed persistent cache service optimized for high performance NVMe SSD.
- dna: The fastest platform to build production-grade indexers that connect onchain data to web2 services.
- si: The System Initiative software.
- AntTP: Serves Autonomi Network data over HTTP protocol.
- Cachey: Read-through cache for object storage.
Quick Start
This section only shows briefs. Please visit https://foyer-rs.github.io/foyer for more details.
To use foyer in your project, add this line to the dependencies section of Cargo.toml.
foyer = "0.22"
If your project is using the nightly rust toolchain, the nightly feature needs to be enabled.
foyer = { version = "0.22", features = ["nightly"] }
Out-of-the-box In-memory Cache
The in-memory cache setup is extremely easy and can be setup in at least 1 line.
use foyer::{Cache, CacheBuilder};
let cache: Cache<String, String> = CacheBuilder::new(16).build();
let entry = cache.insert("hello".to_string(), "world".to_string());
let e = cache.get("hello").unwrap();
assert_eq!(entry.value(), e.value());
Easy-to-use Hybrid Cache
The setup of a hybrid cache is extremely easy.
use foyer::{BlockEngineConfig, DeviceBuilder, FsDeviceBuilder, HybridCache, HybridCacheBuilder};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let dir = tempfile::tempdir()?;
let device = FsDeviceBuilder::new(dir.path())
.with_capacity(256 * 1024 * 1024)
.build()?;
let hybrid: HybridCache<u64, String> = HybridCacheBuilder::new()
.memory(64 * 1024 * 1024)
.storage()
// use block-based disk cache engine with default configuration
.with_engine_config(BlockEngineConfig::new(device))
.build()
.await?;
hybrid.insert(42, "The answer to life, the universe, and everything.".to_string());
assert_eq!(
hybrid.get(&42).await?.unwrap().value(),
"The answer to life, the universe, and everything."
);
Ok(())
}
Fully Configured Hybrid Cache
Here is an example of a hybrid cache setup with almost all configurations to show the possibilities of tuning.
use std::{hash::BuildHasherDefault, num::NonZeroUsize};
use foyer::{
BlockEngineConfig, DeviceBuilder, FifoPicker, FsDeviceBuilder, HybridCache, HybridCacheBuilder, HybridCachePolicy,
IopsCounter, LruConfig, PsyncIoEngineConfig, RecoverMode, RejectAll, StorageFilter, Throttle,
};
use tempfile::tempdir;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let dir = tempdir()?;
let device = FsDeviceBuilder::new(dir.path())
.with_capacity(64 * 1024 * 1024)
.with_throttle(
Throttle::new()
.with_read_iops(4000)
.with_write_iops(2000)
.with_write_throughput(100 * 1024 * 1024)
.with_read_throughput(800 * 1024 * 1024)
.with_iops_counter(IopsCounter::PerIoSize(NonZeroUsize::new(128 * 1024).unwrap())),
)
.build()?;
let hybrid: HybridCache<u64, String> = HybridCacheBuilder::new()
.with_name("my-hybrid-cache")
.with_policy(HybridCachePolicy::WriteOnEviction)
.memory(1024)
.with_shards(4)
.with_eviction_config(LruConfig {
high_priority_pool_ratio: 0.1,
})
.with_hash_builder(BuildHasherDefault::default())
.with_weighter(|_key, value: &String| value.len())
.with_filter(|_, _| true)
.storage()
.with_io_engine_config(PsyncIoEngineConfig::new())
.with_engine_config(
BlockEngineConfig::new(device)
.with_block_size(16 * 1024 * 1024)
.with_indexer_shards(64)
.with_recover_concurrency(8)
.with_flushers(2)
.with_reclaimers(2)
.with_buffer_pool_size(256 * 1024 * 1024)
.with_clean_block_threshold(4)
.with_eviction_pickers(vec![Box::<FifoPicker>::default()])
.with_admission_filter(StorageFilter::new())
.with_reinsertion_filter(StorageFilter::new().with_condition(RejectAll))
.with_tombstone_log(false),
)
.with_recover_mode(RecoverMode::Quiet)
.with_compression(foyer::Compression::Lz4)
.with_spawner(
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.worker_threads(4)
.max_blocking_threads(2)
.build()
.unwrap()
.into(),
)
.build()
.await?;
hybrid.insert(42, "The answer to life, the universe, and everything.".to_string());
assert_eq!(
hybrid.get(&42).await?.unwrap().value(),
"The answer to life, the universe, and everything."
);
let e = hybrid
.get_or_fetch(&20230512, || async {
// Mock fetching data from remote source
let now = jiff::Zoned::now();
if format!("{}{}{}", now.year(), now.month(), now.day()) == "20230512" {
return Err(anyhow::anyhow!("Hi, time traveler!"));
}
Ok("Hello, foyer.".to_string())
})
.await?;
assert_eq!(e.key(), &20230512);
assert_eq!(e.value(), "Hello, foyer.");
hybrid.close().await?;
Ok(())
}
serde Support
foyer needs to serialize/deserialize entries between memory and disk with hybrid cache. Cached keys and values need to implement the Code trait when using hybrid cache.
The Code trait has already been implemented for general types, such as:
- Numeric types:
u8,u16,u32,u64,u128,usize,i8,i16,i32,i64,i128,isize,f32,f64. - Buffer:
Vec<u8>. - String:
String. - Other general types:
bool.
For more complex types, you need to implement the Code trait yourself.
To make things easier, foyer provides support for the serde ecosystem. Types implement serde::Serialize and serde::DeserializeOwned, foyer will automatically implement the Code trait. This feature re
