Nano Benchmark

© KX 2025

"Nano" is a benchmark utility that measures the raw CPU, memory, and I/O performance of a system from the perspective of a kdb+ process. It performs a series of tests including sequential/random reads and writes, data aggregation (e.g., sum, sort), and filesystem metadata operations (e.g., opening a file).

The benchmark can measure performance using a single kdb+ process or aggregate results from multiple worker processes. These processes can run against local storage or a distributed/shared file system. Throughput and latency are measured directly within kdb+/q and cover operations like data creation, ingestion, memory mapping (mmap), and reads. An option to test with compressed data is also included.

It's designed to validate the performance of a system's CPU and storage before proceeding with more comprehensive testing.

While Nano does not simulate a multi-column tick database, it effectively demonstrates the maximum I/O capabilities of kdb+ on the system at a lower level. It is also a valuable tool for examining OS settings, file systems, testing scalability, and identifying I/O-related bottlenecks via stress testing.

Multi-node client testing can be used to test read/write rates for multiple hosts targeting a shared storage device.

Throughout this document, "kdb+ process" and "q process" are used interchangeably.

Prerequisites

• kdb+ 4.1 or later must be installed. You can find installation instructions install kdb+ 4.1. If your kdb+ installation is not in $HOME/q, you must set the QHOME environment variable in config/kdbenv.
• Required command-line utilities: The scripts require the following tools to be available in your PATH. See the Dockerfile for a complete list of dependencies.
  • iostat (from the sysstat package)
  • nc (netcat)
• Network Ports: The scripts launch several kdb+ processes that listen for IPC messages. By default, the controller listens on port 5100, and workers use ports 5500, 5501, 5502, and so on. These can be configured by editing the CONTROLLERPORT and WORKERBASEPORT variables in nano.sh.

Installation and Configuration

Clone or copy all scripts into a single working directory.

Important: Do not place the scripts in the directory you intend to use for I/O testing, as this directory may be unmounted by certain test routines. If running across multiple nodes, this directory should be on a shared file system (e.g., NFS).

For best results, run the scripts as the root user or with a high process priority (e.g., using nice).

Parameters

The benchmark is configured using both environment variables and command-line arguments. Environment variables are set in config/env.

One key environment variable is FLUSH, which must point to a script that flushes the system's storage cache. Sample scripts are provided in the flush/ directory. The default script, directmount.sh, assumes locally attached block storage and uses echo 3 | ${SUDO} tee /proc/sys/vm/drop_caches to clear caches. If you are not running as root (i.e. SUDO=sudo in config/env), your system administrator will need to grant passwordless sudo permissions for this script.

Storage disks

1. Create a dedicated test directory on each storage device or partition you wish to benchmark.
2. Add the absolute paths of these directories to the partitions file, one per line.

Using a single entry in the partitions file is a simple way to test a shared/parallel file system, allowing it to manage data distribution across its storage targets.

Running the Benchmark

Before running, it is recommended to close all cpu, storage and memory-intensive applications. The benchmark does not use more then third of the available free memory and sizes its test files accordingly.

The scripts rely on environment variables defined in the configuration files. Source them before execution:

$ source ./config/kdbenv
$ source ./config/env

nano.sh

This script starts the benchmark on a single host with multiple worker processes. Use the -h flag for a full list of options.

Example Usages:

# Run with worker processes number equal to the thread count of the system
$ ./nano.sh -p $(nproc)

# Run with 8 workers and skip cleaning up the data directories afterward
$ ./nano.sh -p 8 --noclean

# Rerun a read-only test on an existing dataset from a previous run
$ ./nano.sh -p 8 -s readonly --noclean -d 0408_152349

Typical worker counts for scalability testing are 1, 2, 4, 8, 16, 32, 64, ... see multiproc.sh.

multiproc.sh

To analyze how storage and CPU performance scales with an increasing number of parallel requests, use multiproc.sh. This script repeatedly calls nano.sh with different worker counts and aggregates the results into a single summary file. Use --help to see all options.

Example Usages:

# Run tests for 1, 2, 4, 8, 16, and 32 workers, saving the summary to a custom file
$ ./multiproc.sh -o nano_results.psv -l 32

# Run only the CPU tests for 1, 16, and 128 workers
$ ./multiproc.sh -s cpuonly -p "1 16 128"

multihost.sh

For multi-node tests, list the hostnames or IP addresses of the other nodes in the hostlist file. The script will use ssh to start the benchmark on each remote node. Each server will run the same number of worker processes, and no data is shared between processes.

Prerequisites for multi-host testing:

• All nodes must be time-synchronized (e.g., using NTP).
• Passwordless ssh access must be configured from the control node to all hosts in hostlist. You may need to configure ~/.ssh/config.
• If running as a non-root user, then you need sudo permissions on remote hosts.

Example Usage:

$ source ./config/kdbenv
$ source ./config/env
$ ./multihost.sh -p 32

Results and Logs

Results

Results are saved in PSV (pipe-separated values) format in the directory specified by the --resultdir argument. If omitted, results are saved to results/mmdd_HHMMSS.

• HOSTNAME-throughput.psv: This file summarizes the throughput metrics from all workers. More details of some columns:
  • throughput: The total throughput measured from within the kdb+ processes.
  • The read/write throughput reported by iostat is also included for comparison.
  • accuracy: An estimate of the test synchronization error. It is calculated as the maximum start time difference across all workers, divided by the average test duration.

The results are saved as PSV files set by the command line parameter resultdir. If resultdir is not set then results/mmdd_HHMMSS is used.

Log Files

Progress is reported to standard output. Detailed logs for each kdb+ process are available in the logs/mmdd_HHMMSS directory.

Advanced Usage

Object Storage Support

Nano v2.1+ supports benchmarking object storage (S3, GCS, Azure Blob Storage). Simply add the object storage path to your partitions file. Refer to the KX documentation on object storage for setup and required environment variables.

Example partitions entries:

• s3://kxnanotest/firsttest
• gs://kxnanotest/firsttest
• ms://kxnanotest.blob.core.windows.net/firsttest

The execution environment must be configured with the appropriate cloud vendor's CLI tools and credentials, as if you were manually accessing object storage.

Testing Object Storage Cache

You can also measure the performance impact of the local cache. The recommended procedure is:

1. Provide an object storage path in the partitions file.
2. Set a unique subdirectory for the test data: export DBDIR=$(date +%m%d_%H%M%S).
3. Run a full test to populate the object store: ./nano.sh -p $(nproc) --noclean --dbsubdir $DBDIR.
4. Enable the local cache by setting KX_OBJSTR_CACHE_PATH in ./config/env to an empty directory on a fast local disk.
5. Run a read-only test to populate the cache: ./nano.sh -p $(nproc) -s readonly --dbsubdir $DBDIR --noclean.
6. Run the read-only test again to measure performance with a warm cache: ./nano.sh -p $(nproc) -s readonly --dbsubdir $DBDIR.
7. Clean up the local cache directory when finished: source ./config/env; rm -rf ${KX_OBJSTR_CACHE_PATH:?}/objects.

You can also test the cache impact of the object storage library. The recommended way is to

• Provide an object storage path in partitions
• DBDIR=$(date +%m%d_%H%M%S)
• Run a full test to populate data: ./nano.sh --processnr $(nproc) --noclean --dbsubdir $DBDIR.
• Assign an empty directory on a fast local disk to the environment variable KX_OBJSTR_CACHE_PATH in file ./config/env
• Run the test to populate cache: ./nano.sh --processnr $(nproc) --scope readonly --dbsubdir $DBDIR --noclean
• Run the test again to use cache: ./nano.sh --processnr $(nproc) --scope readonly --dbsubdir $DBDIR
• Delete cache files in object storage cache: source ./config/env; rm -rf ${KX_OBJSTR_CACHE_PATH}/objects

The random read test is deterministic; it uses a fixed seed to ensure it reads the same data blocks in the same order on consecutive runs, which is crucial for cache testing.

The THREADNR environment variable can significantly impact random read performance from object storage. See the documentation on secondary threads for details.

Docker Image

A pre-built Docker image is available from KX download portal.

# Pull from GitLab
$ docker pull portal.dl.kx.com/kx-nano:latest

The nano scripts are located at /opt/kx/app inside the container. To run the benchmark, you must mount several host directories into the co