katipo

An HTTP/HTTP2/HTTP3 client library for Erlang built around libcurl-multi and libevent.

Status

Usage

{ok, _} = application:ensure_all_started(katipo).
Pool = api_server,
{ok, _} = katipo_pool:start(Pool, 2, [{pipelining, multiplex}]).
Url = <<"https://example.com">>.
ReqHeaders = [{<<"User-Agent">>, <<"katipo">>}].
Opts = #{headers => ReqHeaders,
         body => <<"0d5cb3c25b0c5678d5297efa448e1938">>,
         connecttimeout_ms => 5000,
         proxy => <<"http://127.0.0.1:9000">>,
         ssl_verifyhost => false,
         ssl_verifypeer => false},
{ok, #{status := 200,
       headers := RespHeaders,
       cookiejar := CookieJar,
       body := RespBody}} = katipo:post(Pool, Url, Opts).

Or passing the entire request as a map

{ok, _} = application:ensure_all_started(katipo).
Pool = api_server,
{ok, _} = katipo_pool:start(Pool, 2, [{pipelining, multiplex}]).
ReqHeaders = [{<<"User-Agent">>, <<"katipo">>}].
Req = #{url => <<"https://example.com">>,
        method => post,
        headers => ReqHeaders,
        body => <<"0d5cb3c25b0c5678d5297efa448e1938">>,
        connecttimeout_ms => 5000,
        proxy => <<"http://127.0.0.1:9000">>,
        ssl_verifyhost => false,
        ssl_verifypeer => false},
{ok, #{status := 200,
       headers := RespHeaders,
       cookiejar := CookieJar,
       body := RespBody}} = katipo:req(Pool, Req).

Why

We wanted a compatible and high-performance HTTP client so took advantage of the 25+ years of development that has gone into libcurl. To allow large numbers of simultaneous connections libevent is used along with the libcurl-multi interface.

Documentation

API

-type method() :: get | post | put | head | options | patch | delete.
katipo_pool:start(Name :: atom(), size :: pos_integer(), PoolOptions :: proplist()).
katipo_pool:stop(Name :: atom()).

katipo:req(Pool :: atom(), Req :: map()).
katipo:Method(Pool :: atom(), URL :: binary()).
katipo:Method(Pool :: atom(), URL :: binary(), ReqOptions :: map()).

Request options

| Option | Type | Default | Notes | |:------------------------|:------------------------------------|:------------|:------------------------------------------------------------------------------------| | headers | [{binary(), iodata()}] | [] | | | cookiejar | opaque (returned in response) | [] | | | body | iodata() | <<>> | | | connecttimeout_ms | pos_integer() | 30000 | docs | | followlocation | boolean() | false | docs | | ssl_verifyhost | boolean() | true | docs | | ssl_verifypeer | boolean() | true | docs | | capath | binary() | undefined | docs | | cacert | binary() | undefined | docs | | ca_cache_timeout | integer() | 86400 | docs curl >= 7.87.0 (0=disable, -1=forever) | | timeout_ms | pos_integer() | 30000 | docs | | dns_cache_timeout | integer() | 60 | docs (0=disable, -1=forever) | | maxredirs | non_neg_integer() | 9 | docs | | proxy | binary() | undefined | docs | | tcp_fastopen | boolean() | false | docs curl >= 7.49.0 | | pipewait | boolean() | true | docs curl >= 7.43.0 | | interface | binary() | undefined | docs | | unix_socket_path | binary() | undefined | docs curl >= 7.40.0 | | doh_url | binary() | undefined | docs curl >= 7.62.0 | | http_version | curl_http_version_none <br> curl_http_version_1_0 <br> curl_http_version_1_1 <br> curl_http_version_2_0 <br> curl_http_version_2tls <br> curl_http_version_2_prior_knowledge <br> curl_http_version_3 | curl_http_version_none | docs HTTP/3 requires curl >= 7.66.0 | | sslversion | sslversion_default <br> sslversion_tlsv1 <br> sslversion_tlsv1_0 <br> sslversion_tlsv1_1 <br> sslversion_tlsv1_2 <br> sslversion_tlsv1_3 | sslversion_default | docs | | sslcert | binary() | undefined | docs | | sslkey | binary() | undefined | docs | | sslkey_blob | binary() (DER format) | undefined | docs curl >= 7.71.0 | | keypasswd | binary() | undefined | docs | | http_auth | basic <br> digest <br> ntlm <br> negotiate | undefined | docs | | username | binary() | undefined | docs | | password | binary() | undefined | docs | | userpwd | binary() | undefined | docs | | verbose | boolean() | false | docs |

Responses

{ok, #{status := pos_integer(),
       headers := headers(),
       cookiejar := cookiejar(),
       body := body()}}

{error, #{code := atom(), message := binary()}}

Pool Options

| Option | Type | Default | Note | |:------------------------|:------------------------------|:-------------|:-----------------------------------------------------------------------------------------------| | pipelining | nothing <br> http1 <br> multiplex | nothing | HTTP pipelining CURLMOPT_PIPELINING | | max_pipeline_length | non_neg_integer() | 100 | | | max_total_connections | non_neg_integer() | 0 (no limit) | docs | | max_concurrent_streams| non_neg_integer() | 100 | docs curl >= 7.67.0 |

Observability

Katipo uses OpenTelemetry for tracing and metrics.

Tracing

Each HTTP request creates a span with the following attributes:

| Attribute | Description | |:----------|:------------| | http.request.method | HTTP method (GET, POST, etc.) | | url.full | Request URL (query string, fragment and userinfo are stripped for security) | | server.address | Target host | | http.response.status_code | Response status code (on success) |

Metrics

The following metrics are recorded:

| Metric | Type | Description | |:-------|:-----|:------------| | http.client.requests | Counter | Number of HTTP requests (with result and http.response.status_code attributes) | | http.client.duration | Histogram | Total request duration (ms) | | http.client.curl_time | Histogram | Curl total time (ms) | | `http.c