Oryx

Oryx is a high-performance .NET cross-platform functional HTTP request handler library for writing HTTP clients and orchestrating web requests in F#.

An SDK for writing HTTP web clients and orchestrating web requests.

This library enables you to write Web and REST clients and SDKs for various APIs and is currently used by the .NET SDK for Cognite Data Fusion (CDF).

Oryx is heavily inspired by the AsyncRx and Giraffe frameworks and applies the same ideas to the client making the web requests. You can think of Oryx as the client equivalent of Giraffe, where the HTTP request processing pipeline starting at the client, going all the way to the server and back again.

Installation

Oryx is available as a NuGet package. To install:

Using Package Manager:

Install-Package Oryx

Using .NET CLI:

dotnet add package Oryx

Or directly in Visual Studio.

Getting Started

open System.Net.Http
open System.Text.Json

open FSharp.Control.TaskBuilder

open Oryx
open Oryx.SystemTextJson.ResponseReader

[<Literal>]
let Url = "https://en.wikipedia.org/w/api.php"

let options = JsonSerializerOptions()

let query term = [
    struct ("action", "opensearch")
    struct ("search", term)
]

let asyncMain argv = task {
    use client = new HttpClient ()
    let request term =
        httpRequest
        |> GET
        |> withHttpClient client
        |> withUrl Url
        |> withQuery (query term)
        |> fetch
        |> json options
        
    let! result = request "F#" |> runAsync
    printfn "Result: %A" result
}

[<EntryPoint>]
let main argv =
    asyncMain().GetAwaiter().GetResult()
    0 // return an integer exit code

Fundamentals

The main building blocks in Oryx are the HttpContext and the HttpHandler. The context contains all the state needed for making the request, and also contains any response metadata such as headers, response code, etc received from the remote server:

type Context = {
    Request: HttpRequest
    Response: HttpResponse
}

The HttpContext is constructed using a pipeline of asynchronous HTTP handlers.

type IHttpNext<'TSource> =
    abstract member OnSuccessAsync: ctx: HttpContext * content: 'TSource -> Task<unit>
    abstract member OnErrorAsync: ctx: HttpContext * error: exn -> Task<unit>
    abstract member OnCancelAsync: ctx: HttpContext -> Task<unit>


type HttpHandler<'TSource> = IHttpNext<'TSource> -> Task<unit>

The relationship can be seen as:

do! handler success error cancel

An HTTP handler (HttpHandler) is a pipeline that uses or subscribes handler success error cancel the given continuations success, error and cancel, and return a Task of unit.

Each HttpHandler usually transforms the HttpRequest, HttpResponse or the content before passing it down the pipeline by invoking the next success continuation. It may also signal an error by invoking error with an exception to fail the processing of the pipeline.

The easiest way to get your head around the Oryx HttpHandler is to think of it as a functional web request processing pipeline. Each handler has the HttpContext and content at its disposal and can decide whether it wants to fail the request calling error, or continue the request by calling the success handler.

HTTP Handlers

The context and content may then be transformed for individual requests using a series of HTTP handlers. HTTP handlers are like lego bricks and may be composed into more complex HTTP handlers. The HTTP handlers included with Oryx are:

• cache - Caches the last result of a given handler, both the context and the content.
• catch - Catches errors and continues using another handler.
• choose - Choose the first handler that succeeds in a list of handlers.
• chunk - Chunks a sequence of HTTP handlers into sequential and concurrent batches.
• concurrent - Runs a sequence of HTTP handlers concurrently.
• empty - Creates a default empty request. You would usually start the chain with this handler.
• fail- Fails the pipeline and pushes an exception downstream.
• fetch - Fetches from remote using the current context
• skip - Handler that skips (ignores) the content and outputs unit.
• get - Retrieves the content (for use in http builder)
• log - Log information about the given request.
• map - Map the content of the HTTP handler.
• panic - Fails the pipeline and pushes an exception downstream. This error cannot be catched or skipped.
• parse - Parse response stream to a user-specified type synchronously.
• parseAsync - Parse response stream to a user-specified type asynchronously.
• sequential - Runs a sequence of HTTP handlers sequentially.
• singleton - Handler that produces a single content value.
• validate - Validate content using a predicate function.
• withBearerToken - Adds an Authorization header with Bearer token.
• withCancellationToken - Adds a cancellation token to use for the context. This is particularly useful when using Oryx together with C# client code that supplies a cancellation token.
• withContent - Add HTTP content to the fetch request
• withMetrics - Add and IMetrics interface to produce metrics info.
• withError - Detect if the HTTP request failed, and then fail processing.
• withHeader - Adds a header to the context.
• withHeaders - Adds headers to the context.
• withHttpClient - Adds the HttpClient to use for making requests using the fetch handler.
• withHttpClientFactory - Adds an HttpClient factory function to use for producing the HttpClient.
• withLogger - Adds an ILogger for logging requests and responses.
• withLogLevel - The log level (LogLevel) that the logging should be performed at. Oryx will disable logging for LogLevel.None and this is also the default log level.
• withLogFormat - Specify the log format of the log messages written.
• withLogMessage - Log information about the given request supplying a user-specified message.
• withMethod - with HTTP method. You can use GET, PUT, POST instead.
• withQuery - Add URL query parameters
• withResponseType - Sets the Accept header of the request.
• withTokenRenewer - Enables refresh of bearer tokens without building a new context.
• withUrl - Use the given URL for the request.
• withUrlBuilder - Use the given URL builder for the request.
• withUrlBuilder - Adds the URL builder to use. An URL builder constructs the URL for the Request part of the context.

In addition there are several extension for decoding JSON and Protobuf responses:

• json - Decodes the given application/json response into a user-specified type.
• protobuf - - Decodes the given application/protobuf response into a Protobuf specific type.

See JSON and Protobuf Content Handling for more information.

HTTP verbs

The HTTP verbs are convenience functions using the withMethod under the hood:

• GET - HTTP get request
• PUT - HTTP put request
• POST - HTTP post request
• DELETE - HTTP delete request
• OPTIONS - HTTP options request

Composition

The real magic of Oryx is composition. The fact that everything is an HttpHandler makes it easy to compose HTTP handlers together. You can think of them as Lego bricks that you can fit together. Two or more HttpHandler functions may be composed together using the pipelining, i.e using the |> operator. This enables you to compose your web requests and decode the response, e.g as we do when listing Assets in the Cognite Data Fusion SDK:

    let list (query: AssetQuery) (source: HttpHandler<unit>) : HttpHandler<ItemsWithCursor<AssetReadDto>> =
        let url = Url +/ "list"

        source
        |> POST
        |> withVersion V10
        |> withResource url
        |> withContent (() -> new JsonPushStreamContent<AssetQuery>(query, jsonOptions))
        |> fetch
        |> withError decodeError
        |> json jsonOptions

The function list is now also an HttpHandler and may be composed with other handlers to create complex chains for doing multiple sequential or concurrent requests to a web service. And you can do this without having to worry about error handling.

Retrying Requests

Since Oryx is based on HttpClient from System.Net.Http, you may also use Polly for handling resilience.

Concurrent and Sequential Handlers

A sequential operator for running a list of HTTP handlers in sequence.

val sequential:
    handlers     : seq<HttpHandler<'TResult>>
                -> HttpHandler<list<'TResult>>

And a concurrent operator that runs a list of HTTP handlers in parallel.

val concurrent:
    handlers     : seq<HttpHandler<'TResult>>
                -> HttpHandler<list<'TResult>>

You can also combine sequential and concurrent requests by chunking the request. The chunk handler uses chunkSize and maxConcurrency to decide how much will be done in parallel. It takes a list of items and a handler that transform