Http Cache Headers Middleware for ASP.NET Core

ASP.NET Core middleware that adds HttpCache headers to responses (Cache-Control, Expires, ETag, Last-Modified), and implements cache expiration & validation models. It can be used to ensure caches correctly cache responses and/or to implement concurrency for REST-based APIs using ETags.

The middleware itself does not store responses. What it does is generate the correct cache-related headers, and ensure a cache can check for expiration (304 Not Modified) & preconditions (412 Precondition Failed) (often used for concurrency checks). For more information on caches, the different models and the related headers, have a look at https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Caching.

This middleware can be used together with a shared cache, a private cache or both. For production scenarios the best approach is to use this middleware to generate the ETags, combined with a cache server or CDN to inspect those tags and effectively cache the responses. In the sample, the Microsoft.AspNetCore.ResponseCaching cache store is used to cache the responses.

Installation (NuGet)

Install-Package Marvin.Cache.Headers

Usage

First, register the services with ASP.NET Core's dependency injection container (in the ConfigureServices method on the Startup class)

services.AddHttpCacheHeaders();

Then, add the middleware to the request pipeline. Starting with version 6.0, the middleware MUST be added between UseRouting() and UseEndpoints().

app.UseRouting(); 

app.UseHttpCacheHeaders();

app.UseEndpoints(...);

Configuring Options

The middleware allows customization of how headers are generated. The AddHttpCacheHeaders() method has parameters for configuring options related to expiration, validation and middleware.

For example, this code will set the max-age directive to 600 seconds, add the must-revalidate directive and ignore header generation for all responses with status code 500.

services.AddHttpCacheHeaders(
    expirationModelOptions =>
    {
        expirationModelOptions.MaxAge = 600;
    },
    validationModelOptions =>
    {
        validationModelOptions.MustRevalidate = true;
    },
    middlewareOptions => 
    {
        middlewareOptions.IgnoreStatusCodes = new[] { 500 };
    });

There are some predefined collections with status codes you can use when you want to ignore:

all server errors HttpStatusCodes.ServerErrors
all client errors HttpStatusCodes.ClientErrors
all errors HttpStatusCodes.AllErrors

Action (Resource) and Controller-level Header Configuration

For anything but the simplest of cases having one global cache policy isn't sufficient: configuration at level of each resource (action/controller) is required. For those cases, use the HttpCacheExpiration and/or HttpCacheValidation attributes at action or controller level.

[HttpGet]
[HttpCacheExpiration(CacheLocation = CacheLocation.Public, MaxAge = 99999)]
[HttpCacheValidation(MustRevalidate = true)]
public IEnumerable<string> Get()
{
    return new[] { "value1", "value2" };
}
```
Both override the global options.  Action-level configuration overrides controller-level configuration.

# Ignoring Cache Headers / eTag Generation

You don't always want tags / headers to be generated for all resources (e.g.: for a large file).  You can ignore generation by applying the HttpCacheIgnore attribute at controller or action level. 

````csharp
[HttpGet]
[HttpCacheIgnore]
public IEnumerable<string> Get()
{
    return new[] { "value1", "value2" };
}

If you want to globally disable automatic header generation, you can do so by setting DisableGlobalHeaderGeneration on the middleware options to true.

services.AddHttpCacheHeaders(     
    middlewareOptionsAction: middlewareOptions => 
    {
        middlewareOptions.DisableGlobalHeaderGeneration = true;
    });

Marking for Invalidation

Cache invalidation essentially means wiping a response from the cache because you know it isn't the correct version anymore. Caches often partially automate this (a response can be invalidated when it becomes stale, for example) and/or expose an API to manually invalidate items.

The same goes for the cache headers middleware, which holds a store of records with previously generated cache headers & tags. Replacement of store key records (/invalidation) is mostly automatic. Say you're interacting with values/1. First time the backend is hit and you get back an eTag in the response headers. Next request you send is again a GET request with the "If-None-Match"-header set to the eTag: the backend won't be hit. Then, you send a PUT request to values/1, which potentially results in a change; if you send a GET request now, the backend will be hit again.

However: if you're updating/changing resources by using an out of band mechanism (eg: a backend process that changes the data in your database, or a resource gets updated that has an update of related resources as a side effect), this process can't be automated.

Take a list of employees as an example. If a PUT statement is sent to one "employees" resource, then that one "employees" resource will get a new Etag. Yet: if you're sending a PUT request to one specific employee ("employees/1", "employees/2", ...), this might have the effect that the "employees" resource has also changed: if the employee you just updated is one of the employees in the returned employees list when fetching the "employees" resource, the "employees" resource is out of date. Same goes for deleting or creating an employee: that, too, might have an effect on the "employees" resource.

To support this scenario the cache headers middleware allows marking an item for invalidation. When doing that, the related item will be removed from the internal store, meaning that for subsequent requests a stored item will not be found.

To use this, inject an IValidatorValueInvalidator and call MarkForInvalidation on it, passing through the key(s) of the item(s) you want to be removed. You can additionally inject an IStoreKeyAccessor, which contains methods that make it easy to find one or more keys from (part of) a URI.

Extensibility

The middleware is very extensible. If you have a look at the AddHttpCacheHeaders method you'll notice it allows injecting custom implementations of IValidatorValueStore, IStoreKeyGenerator, IETagGenerator and/or IDateParser (via actions).

IValidatorValueStore

A validator value store stores validator values. A validator value is used by the cache validation model when checking if a cached item is still valid. It contains ETag and LastModified properties. The default IValidatorValueStore implementation (InMemoryValidatorValueStore) is an in-memory store that stores items in a ConcurrentDictionary<string, ValidatorValue>.

/// <summary>
/// Contract for a store for validator values.  Each item is stored with a <see cref="StoreKey" /> as key```
/// and a <see cref="ValidatorValue" /> as value (consisting of an ETag and Last-Modified date).   
/// </summary>
public interface IValidatorValueStore
{
    /// <summary>
    /// Get a value from the store.
    /// </summary>
    /// <param name="key">The <see cref="StoreKey"/> of the value to get.</param>
    /// <returns></returns>
    Task<ValidatorValue> GetAsync(StoreKey key);
    /// <summary>
    /// Set a value in the store.
    /// </summary>
    /// <param name="key">The <see cref="StoreKey"/> of the value to store.</param>
    /// <param name="validatorValue">The <see cref="ValidatorValue"/> to store.</param>
    /// <returns></returns>
    Task SetAsync(StoreKey key, ValidatorValue validatorValue);

    /// <summary>
    /// Find one or more keys that contain the inputted valueToMatch 
    /// </summary>
    /// <param name="valueToMatch">The value to match as part of the key</param>
    /// <param name="ignoreCase">Ignore case when matching</param>
    /// <returns></returns>
    Task<IEnumerable<StoreKey>> FindStoreKeysByKeyPartAsync(string valueToMatch, bool ignoreCase);
}

BREAKING CHANGE from v7 onwards: the FindStoreKeysByKeyPartAsync methods return an IAsyncEnumerable<StoreKey> to enable async streaming of results.

/// <summary>
/// Contract for a store for validator values.  Each item is stored with a <see cref="StoreKey" /> as key```
/// and a <see cref="ValidatorValue" /> as value (consisting of an ETag and Last-Modified date).   
/// </summary>
public interface IValidatorValueStore
{
    /// <summary>
    /// Get a value from the store.
    /// </summary>
    /// <param name="key">The <see cref="StoreKey"/> of the value to get.</param>
    /// <returns></returns>
    Task<ValidatorValue> GetAsync(StoreKey key);
    /// <summary>
    /// Set a value in the store.
    /// </summary>
    /// <param name="key">The <see cref="StoreKey"/> of the value to store.</param>
    /// <param name="validatorValue">The <see cref="ValidatorValue"/> to store.</param>
    /// <returns></returns>
    Task SetAsync(StoreKey key, ValidatorValue validatorValue);

    /// <summary>
    /// Find one or more keys that contain the inputted valueToMatch 
    /// </summary>
    /// <param name="valueToMatch">The value to match as part of the key</param>
    /// <param name="ignoreCase">Ignore case when matching</param>
    /// <returns></returns>
    IAsyncEnumerable<StoreKey> FindStoreKeysByKeyPartAsync(string valueToMatch, bool ignoreCase);
}

IStoreKeyGenerator

The StoreKey, as used by the IValidatorValueStore as key, can be customized as well. To do so, implement the IStoreKeyGenerator interface. The default implementation (DefaultStoreKeyGenerator) generates a key from the request path, request query string and request header valu

HttpCacheHeaders

Install / Use

README