Hitbox

Highly customizable async caching framework for Rust designed for high-performance applications.

Protocol-agnostic async core + first-class HTTP support via hitbox-http. Pluggable backends from in-memory to distributed solutions such as Redis. Built on tower, works with any tokio-based service.

There are several common approaches to caching. The first is a low-level approach where you work directly with a cache instance, calling get, set, and delete methods and deciding exactly when and where to use them in your code. The second is a middleware approach where a caching layer wraps your handlers or services transparently, keeping your business logic clean and business-focused. The third is function-level memoization — like Python decorators — where you simply annotate a function and work with it as if no caching exists. Hitbox supports the second and third approaches, keeping caching logic out of your business code in both cases.

While Hitbox is designed for large, high-load projects, it works equally well for small and simple ones. The configuration complexity scales with your project: simple projects need only simple settings.

• Motivation
• Quick Start
• Features
• Project Structure
• Benchmarks
• License

Motivation

Every real-world system brings a combination of shared challenges and unique constraints. We tried using existing caching frameworks for our services, but each time they failed to fully match our requirements. As a result, we repeatedly ended up building custom caching mechanisms from scratch instead of relying on ready-made solutions.

Hitbox was created to break this cycle.

We think of Hitbox not as a library, but as a platform for caching, designed from day one to be easily extensible without enforcing a single backend, protocol, or caching strategy. New storage backends, new protocols such as GraphQL or even the PostgreSQL protocol, as well as new serialization or compression strategies, are expected use cases - not special exceptions. You simply implement the required traits and go for it. Hitbox is built to be hacked, extended, bent, and reshaped.

A key principle of Hitbox is that every new integration automatically inherits the full set of advanced optimizations we built through real-world experience: dogpile-effect prevention, composable multi-layer caching (L1/L2/L3), offload caching, and more. Instead of re-implementing these mechanisms for every project, they come for free with the platform.

At the same time, Hitbox is not just an abstract foundation. It already provides a production-ready HTTP caching implementation based on tower::Service, covering the most common use case out of the box while also serving as a reference implementation for building additional integrations.

---

Quick Start

HTTP Middleware (Axum / Tower)

Cargo.toml

[package]
name = "hitbox-example"
version = "0.1.0"
edition = "2024"

[dependencies]
axum = "0.8"
tokio = { version = "1", features = ["full"] }
hitbox = "0.2"
hitbox-http = "0.2"
hitbox-moka = "0.2"
hitbox-tower = "0.2"
http = "1"

Basic Usage

use std::time::Duration;

use axum::{Router, extract::Path, routing::get};
use hitbox::policy::PolicyConfig;
use hitbox::{Config, Neutral};
use hitbox_http::extractors::Method as MethodExtractor;
use hitbox_http::extractors::path::PathExtractor;
use hitbox_http::predicates::request::Method;
use hitbox_http::predicates::response::{StatusClass, StatusCodePredicate};
use hitbox_moka::MokaBackend;
use hitbox_tower::Cache;

async fn get_users() -> &'static str {
    "users list"
}
async fn get_user(Path(id): Path<String>) -> String {
    format!("user {id}")
}

#[tokio::main]
async fn main() {
    // Create backend
    let backend = MokaBackend::builder().max_entries(10_000).build();

    // Users list - long TTL (60s)
    let users_config = Config::builder()
        .request_predicate(Method::new(http::Method::GET).unwrap())
        .response_predicate(Neutral::new().status_code_class(StatusClass::Success))
        .extractor(MethodExtractor::new().path("/api/users"))
        .policy(
            PolicyConfig::builder()
                .ttl(Duration::from_secs(60))
                .stale(Duration::from_secs(30))
                .build(),
        )
        .build();

    // Single user - short TTL (10s)
    let user_config = Config::builder()
        .request_predicate(Method::new(http::Method::GET).unwrap())
        .response_predicate(Neutral::new().status_code_class(StatusClass::Success))

        .extractor(MethodExtractor::new().path("/api/users/{id}"))
        .policy(
            PolicyConfig::builder()
                .ttl(Duration::from_secs(10))
                .stale(Duration::from_secs(5))
                .build(),
        )
        .build();

    // Build cache layers
    let users_cache = Cache::builder()
        .backend(backend.clone())
        .config(users_config)
        .build();

    let user_cache = Cache::builder()
        .backend(backend)
        .config(user_config)
        .build();

    // Router with per-route cache layers
    let app = Router::new()
        .route("/api/users", get(get_users).layer(users_cache))
        .route("/api/users/{id}", get(get_user).layer(user_cache));


    println!("Starting server on http://localhost:3000");
    println!("Try:");
    println!("  curl -v http://localhost:3000/api/users");
    println!("  curl -v http://localhost:3000/api/users/42");

    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}

Function Memoization

The hitbox-fn crate provides function-level memoization — cache async function results without touching your business logic. Annotate functions with #[cached], derive KeyExtract on argument types to control cache key generation, and derive CacheableResponse on return types to control what gets stored. Build a reusable Cache instance with a backend and policy, then call your function via .cache(&cache). Without cache configuration, #[cached] functions can be called directly with .await — a transparent passthrough to the underlying function. Enable the derive feature flag for macro support.

See the memoization_derive example for a complete walkthrough including multi-argument functions, skipped fields, and CacheableResponse with excluded fields.

What's Next

• Predicates - Control what gets cached
• Stale Cache - Configure TTL and background revalidation
• Composable Backends - Add Redis as L2
• Examples - More complete examples
• Blog - Find useful information on using Hitbox

---

Features

Core Features

• Stale Cache Serve stale data within grace period while revalidating in background
• Dogpile Prevention Configurable concurrency limit with broadcast to prevent redundant upstream calls
• Pluggable Backends Choose Moka, Redis, FeOxDB, or implement the Backend trait for your own storage
• Composable Backends Combine backends into L1/L2/L3 tiers for optimal performance
• Serialization Choose between bincode, rkyv, JSON, or RON formats
• Compression Reduce storage size with zstd or gzip compression
• Observability Track cache status, latency, backend I/O, and offload tasks
• Predicate and Extractor Traits Protocol-agnostic traits to control caching and generate cache keys
• Function Memoization Cache async function results with #[cached] macro and derive-based cache key extraction

HTTP Caching Features

• HTTP Predicates Control caching with rules based on any part of request or response, including body
• HTTP Extractors Automatically generate cache keys from request components
• Framework Integration Works with Axum, Hyper client, Reqwest, and any tower-based framework or client
• YAML Configuration Define entire caching setup in a configuration file

---

Stale Cache

Stale cache enables serving outdated data while refreshing in the background. Each cache entry has a TTL (fresh period) and an optional stale window (grace period). During TTL, data is fresh. After TTL but within the stale window, data is stale but servable. Three policies control stale behavior: Return serves stale data immediately without revalidation, Revalidate blocks until fresh data is fetched, and OffloadRevalidate serves stale data immediately while refreshing in the background (Stale-While-Revalidate pattern). The OffloadManager handles background revalidation with task deduplication, configurable timeouts (none, cancel, or warn), and metrics tracking.

OffloadManager Configuration

| Option | Description | Default | |--------|-------------|---------| | max_concurrent_tasks | Limit parallel background tasks | Unlimited | | timeout_policy | None, Cancel(duration), or Warn(duration) | None | | deduplicate | Prevent duplicate revalidation for same key | true |

**Code examp