ForkUnion

ForkUnion is a NUMA-aware fork-join thread-pool for C++, C, Rust, and Zig — built for the tightest #pragma omp parallel for-style loops, not task queues. 🍴 It already powers NUMA-sharded vector search with USearch, LLM KV-cache and attention kernels with NumKong, and unbalanced bioinformatics workloads — thousands of combinatorial tasks per core — with StringZilla.

On the hot path it makes zero heap allocations, zero system calls, zero CAS operations, and suffers no false-sharing of cache-lines. So dispatch latency stays flat into the hundreds of cores, precisely where task-queue runtimes like Rayon collapse and even OpenMP begins to slip. It is also unique in parking idle workers on a hardware address monitor — x86 UMONITOR/UMWAIT, Arm WFET, RISC-V Zawrs — so a worker light-sleeps on the exact cache-line it is waiting for and the silicon wakes it the instant another core writes there, with no hot spinning and no kernel futex. That bet compounds as sockets multiply: Intel Xeon Platinum and NVIDIA Vera pack up to 8 sockets per node, where shared CAS thrashes the interconnect and private fetch_add cursors keep synchronization off it.

One parallel-for dispatch: ForkUnion is often ~3–6× faster than OpenMP and ~12–16× faster than Rayon and Taskflow — the fork-join tax, paid once per loop, cut by an order of magnitude. Full tables ↓

It is exhaustively tested for boundary-condition scheduling with miniaturized uint8_t indices, even runs on your big-endian 32-bit IBM mainframe, and ships with no_std and Miri coverage. The core is a C++ 17 library; the C 99, Rust, and Zig APIs bind it, and all four can pin threads to NUMA nodes or individual cores and allocate node-local memory. Despite being far more deeply tied to hardware and the OS than most alternatives, ForkUnion runs on six operating systems — Linux, FreeBSD, Windows, macOS, Android, and iOS — including asymmetric compute and memory topologies. Topology harvesting and thread placement work on all six; NUMA-local memory placement is implemented on Linux, FreeBSD, and Windows, and elsewhere allocation falls back to a single memory domain.

Basic Usage

ForkUnion is dead-simple to use! There is no nested parallelism, exception handling, or "future promises"; they are banned. The thread pool itself has a few core operations:

• try_spawn to initialize worker threads, and
• for_threads to launch a blocking callback on all threads.

Higher-level APIs for index-addressable tasks are also available:

• for_n - for individual evenly-sized tasks,
• for_n_dynamic - for individual unevenly-sized tasks,
• for_slices - for slices of evenly-sized tasks.

For additional flow control and tuning, following helpers are available:

• sleep(microseconds) - for longer naps,
• terminate - to kill the threads before the destructor is called,
• unsafe_for_threads - to broadcast a callback without blocking, returning a generation token,
• unsafe_join - to block until the completion of a broadcasted generation,
• is_complete - to poll a generation token for completion without blocking.

Every dispatch is identified by an always-odd generation token. On caller_exclusive_k pools you can dispatch, overlap your own work, poll is_complete, and join - the classic poll-then-join pattern. On caller_inclusive_k pools the calling thread owes one slice of the work, which only runs inside unsafe_join, so completion can't be reached by polling alone. The same rule shapes the RAII guard returned by for_threads and the for_n family: on exclusive pools the work starts at the guard's construction, while on inclusive pools it runs at join or destruction.

On Linux, in C++, given the maturity and flexibility of the HPC ecosystem, it provides NUMA extensions. That includes the colocated_pool analog of the flat_pool and the linux_numa_allocator for allocating memory in a specific memory domain. Those are out-of-the-box compatible with the higher-level APIs. Most interestingly, for Big Data applications, a higher-level distributed_pool class will address and balance the work across all compute domains.

Intro in Rust

To integrate into your Rust project, add the following lines to Cargo.toml:

[dependencies]
forkunion = "3.0.2"                                          # detect what the platform offers
forkunion = { version = "3.0.2", features = ["portable"] }   # STL thread pool only
forkunion = { version = "3.0.2", features = ["place-memory-on-domain"] } # require NUMA-aware allocations

Or for the preview development version:

[dependencies]
forkunion = { git = "https://github.com/ashvardanian/ForkUnion.git", branch = "main-dev" }

A minimal example may look like this:

use forkunion as fu;
let topology = fu::Topology::new().expect("Failed to detect hardware topology");
let mut pool = fu::spawn(&topology, 2);
pool.for_threads(&|thread_index, compute_domain_index| {
    println!("Hello from thread # {} on compute domain # {}", thread_index + 1, compute_domain_index + 1);
});

Higher-level APIs distribute index-addressable tasks across the threads in the pool:

pool.for_n(100, |prong| {
    println!("Running task {} on thread # {}",
        prong.task_index + 1, prong.thread_index + 1);
});
pool.for_slices(100, |prong, count| {
    println!("Running slice [{}, {}) on thread # {}",
        prong.task_index, prong.task_index + count, prong.thread_index + 1);
});
pool.for_n_dynamic(100, |prong| {
    println!("Running task {} on thread # {}",
        prong.task_index + 1, prong.thread_index + 1);
});

A more realistic example with named threads and error handling may look like this:

use std::error::Error;
use forkunion as fu;

fn heavy_math(_: usize) {}

fn main() -> Result<(), Box<dyn Error>> {
    let topology = fu::Topology::new()?;
    let mut pool = fu::ThreadPool::try_named_spawn(&topology, "heavy-math", 4)?;
    pool.for_n_dynamic(400, |prong| {
        heavy_math(prong.task_index);
    });
    Ok(())
}

The Topology handle reports the hardware topology, to size and place work:

let topology = fu::Topology::new()?;
for domain in 0..topology.compute_domains_count() {
    let domain = fu::ComputeDomain(domain);
    println!("domain {}: {} cores, level {}, allocate on memory domain {}",
        domain.get(), topology.logical_cores_count_in(domain),
        topology.compute_level_in(domain), topology.local_memory_of(domain).get());
}

For advanced usage, refer to the NUMA section below. For convenience Rayon-style parallel iterators pull the prelude module and check out related examples.

Intro in C++

To integrate into your C++ project, either copy the include/ directory into your project, add a Git submodule, or CMake. The forkunion.hpp umbrella is a list of #includes; the implementation lives beside it in include/forkunion/, one header per concern, so a platform backend can be read or replaced on its own. For a Git submodule, run:

git submodule add https://github.com/ashvardanian/ForkUnion.git extern/forkunion

Alternatively, using CMake:

FetchContent_Declare(
    forkunion
    GIT_REPOSITORY https://github.com/ashvardanian/ForkUnion
    GIT_TAG v3.0.2
)
FetchContent_MakeAvailable(forkunion)
target_link_libraries(your_target PRIVATE forkunion::header)

Then, include the header in your C++ code:

#include <forkunion.hpp>    // `flat_pool_t`
#include <cstdio>           // `stderr`
#include <cstdlib>          // `EXIT_SUCCESS`

namespace fu = ashvardanian::forkunion;

int main() {
    alignas(fu::default_alignment_k) fu::flat_pool_t pool;
    if (!pool.try_spawn(fu::allowed_cores_count())) {
        std::fprintf(stderr, "Failed to fork the threads\n");
        return EXIT_FAILURE;
    }

    // Dispatch a callback to each thread in the pool
    pool.for_threads([&](std::size_t thread_index) noexcept {
        std::printf("Hello from thread # %zu (of %zu)\n", thread_index + 1, pool.threads_count());
    });

    // Execute 1000 tasks in parallel, expecting them to have comparable runtimes
    // and mostly co-locating subsequent tasks on the same thread. Analogous to:
    //
    //      #pragma omp parallel for schedule(static)
    //      for (int i = 0; i < 1000; ++i) { ... }
    //
    // You can also think about it as a shortcut for the `for_slices` + `for`.
    pool.for_n(1000, [](std::size_t task_index) noexcept {
        std::printf("Running task %zu of 1000\n", task_index + 1);
    });
    pool.for_slices(1000, [](std::size_t first_index, std::size_t count) noexcept {
        std::printf("Running slice [%zu, %zu)\n", first_index, first_index + count);
    });

    // Like `for_n`, but each thread greedily steals tasks, without waiting for  
    // the others or expecting individual tasks to have same runtimes. Analogous to:
    //
    //      #pragma omp parallel for schedule(dynamic, 1)
    //      for (int i = 0; i < 3; ++i) { ... }
    pool.for_n_dynamic(3, [](std::size_t task_index) noexcept {
        std::printf("Running dynamic task %zu of 3\n", task_index + 1);
    });
    return EXIT_SUCCESS;
}

For advanced usage, refer to the NUMA section below. Every kernel and ISA facility the library uses is detected by default. CMake pins ea