[API reference][Eio API] | #eio Matrix chat | [Dev meetings][]

Eio — Effects-Based Parallel IO for OCaml

Eio provides an effects-based direct-style IO stack for OCaml 5. For example, you can use Eio to read and write files, make network connections, or perform CPU-intensive calculations, running multiple operations at the same time. It aims to be easy to use, secure, well documented, and fast. A generic cross-platform API is implemented by optimised backends for different platforms. Eio replaces existing concurrency libraries such as Lwt (Eio and Lwt libraries can also be used together).

Contents

<!-- vim-markdown-toc GFM -->

• Motivation
• Eio packages
• Getting OCaml
• Getting Eio
• Running Eio
• Testing with Mocks
• Fibers
• Tracing
• Cancellation
• Racing
• Switches
• Networking
• Design Note: Capabilities
• Buffered Reading and Parsing
• Buffered Writing
• Error Handling
• Filesystem Access
• Running processes
• Time
• Multicore Support
  • Domain Manager
  • Executor Pool
• Synchronisation Tools
  • Promises
  • Example: Concurrent Cache
  • Streams
  • Example: Worker Pool
  • Mutexes and Semaphores
  • Conditions
  • Example: Signal handlers
• Design Note: Determinism
• Provider Interfaces
• Example Applications
• Integrations
  • Async
  • Lwt
  • Unix and System Threads
  • Domainslib
  • kcas
• Best Practices
  • Switches
  • Casting
  • Passing env
• Further Reading

<!-- vim-markdown-toc -->

Motivation

The Unix library provided with OCaml uses blocking IO operations, and is not well suited to concurrent programs such as network services or interactive applications. For many years, the solution was to use libraries such as Lwt and Async, which provide a monadic interface. These libraries allow writing code as if there were multiple threads of execution, each with their own stack, but the stacks are simulated using the heap.

OCaml 5 added support for "effects", removing the need for monadic code here. Using effects brings several advantages:

1. It's faster, because no heap allocations are needed to simulate a stack.
2. Concurrent code can be written in the same style as plain non-concurrent code.
3. Because a real stack is used, backtraces from exceptions work as expected.
4. Other features of the language (such as try ... with ...) can be used in concurrent code.

Additionally, modern operating systems provide high-performance alternatives to the old Unix select call. For example, Linux's io_uring system has applications write the operations they want to perform to a ring buffer, which Linux handles asynchronously, and Eio can take advantage of this.

You can always fall back to using Lwt libraries to provide missing features if necessary. See [Awesome Multicore OCaml][] for links to other projects using Eio.

Eio packages

• [Eio][] provides concurrency primitives (promises, etc.) and a high-level, cross-platform OS API.
• [Eio_posix][] provides a cross-platform backend for these APIs for POSIX-type systems.
• [Eio_linux][] provides a Linux io_uring backend for these APIs.
• [Eio_windows][] is for use on Windows (incomplete - help wanted).
• [Eio_main][] selects an appropriate backend (e.g. eio_linux or eio_posix), depending on your platform.
• [Eio_js][] allows Eio code to run in the browser, using js_of_ocaml.

Getting OCaml

You'll need OCaml 5.1.0 or later. You can either install it yourself or build the included Dockerfile.

To install it yourself:

1. Make sure you have opam 2.1 or later (run opam --version to check).

2. Use opam to install OCaml:

   opam switch create 5.2.0
   

Getting Eio

Install eio_main (and utop if you want to try it interactively):

opam install eio_main utop

If you want to install the latest unreleased development version of Eio, see HACKING.md.

Running Eio

Try out the examples interactively by running utop in the shell.

First require the eio_main library. It's also convenient to open the [Eio.Std][] module, as follows. (The leftmost # shown below is the Utop prompt, so enter the text after the prompt and return after each line.)

# #require "eio_main";;
# open Eio.Std;;

This function writes a greeting to out using [Eio.Flow][]:

let main out =
  Eio.Flow.copy_string "Hello, world!\n" out

We use [Eio_main.run][] to run the event loop and call main from there:

# Eio_main.run @@ fun env ->
  main (Eio.Stdenv.stdout env);;
Hello, world!
- : unit = ()

Note that:

• The env argument represents the standard environment of a Unix process, allowing it to interact with the outside world. A program will typically start by extracting from env whatever things the program will need and then calling main with them.

• The type of the main function here tells us that this program only interacts via the out flow.

• Eio_main.run automatically calls the appropriate run function for your platform. For example, on Linux this will call Eio_linux.run. For non-portable code you can use the platform-specific library directly.

This example can also be built using dune; see examples/hello.

Testing with Mocks

Because external resources are provided to main as arguments, we can easily replace them with mocks for testing. For example, instead of giving main the real standard output, we can have it write to a buffer:

# Eio_main.run @@ fun _env ->
  let buffer = Buffer.create 20 in
  main (Eio.Flow.buffer_sink buffer);
  traceln "Main would print %S" (Buffer.contents buffer);;
+Main would print "Hello, world!\n"
- : unit = ()

[Eio.traceln][] provides convenient printf-style debugging, without requiring you to plumb stderr through your code. It uses the Format module, so you can use the extended formatting directives here too.

The [Eio_mock][] library provides some convenient pre-built mocks:

# #require "eio.mock";;
# Eio_main.run @@ fun _env ->
  main (Eio_mock.Flow.make "mock-stdout");;
+mock-stdout: wrote "Hello, world!\n"
- : unit = ()

Fibers

Here's an example running two threads of execution concurrently using [Eio.Fiber][]:

let main _env =
  Fiber.both
    (fun () -> for x = 1 to 3 do traceln "x = %d" x; Fiber.yield () done)
    (fun () -> for y = 1 to 3 do traceln "y = %d" y; Fiber.yield () done);;

# Eio_main.run main;;
+x = 1
+y = 1
+x = 2
+y = 2
+x = 3
+y = 3
- : unit = ()

The two fibers run on a single core, so only one can be running at a time. Calling an operation that performs an effect (such as yield) can switch to a different thread.

Tracing

When OCaml's tracing is turned on, Eio writes events about many actions, such as creating fibers or resolving promises.

You can use [eio-trace][] to capture a trace and display it in a window. For example, this is a trace of the counting example above:

dune build ./examples
eio-trace run -- ./_build/default/examples/both/main.exe

<p align='center'> <img src="./doc/traces/both-posix.svg"/> </p>

The upper horizontal bar is the initial fiber, and the brackets show Fiber.both creating a second fiber. The green segments show when each fiber is running. Note that the output from traceln appears in the trace as well as on the console. In the eio-trace window, scrolling with the mouse or touchpad will zoom in or out of the diagram.

Third-party tools, such as [Olly][], can also consume this data. examples/trace shows how to consume the events manually.

Cancellation

Every fiber has a [cancellation context][Eio.Cancel]. If one of the Fiber.both fibers fails, the other is cancelled:

# Eio_main.run @@ fun _env ->
  Fiber.both
    (fun () -> for x = 1 to 3 do traceln "x = %d" x; Fiber.yield () done)
    (fun () -> failwith "Simulated error");;
+x = 1
Exception: Failure "Simulated error".

<p align='center'> <img src="./doc/traces/cancel-posix.svg"/> </p>

What happened here was:

1. Fiber.both created a new cancellation context for the child fibers.
2. The first fiber (the lower one in the diagram) ran, printed x = 1 and yielded.
3. The second fiber raised an exception.
4. Fiber.both caught the exception and cancelled the context.
5. The first thread's yield raised a Cancelled exception there.
6. Once both threads had finished, Fiber.both re-raised the original exception.

There is a tree of cancellation contexts for each domain, and every fiber is in one context. When an exception is raised, it propagates towards the root until handled, cancelling the other branches as it goes. You should assume that any operation that can switch fibers can also raise a Cancelled exception if an uncaught exception reaches one of its ancestor cancellation contexts.

If you want to make an operation non-cancellable, wrap it with Cancel.protect (this creates a new context that isn't cancelled with its parent).

Racing

Fiber.first returns the result of the first fiber to fini