Llfix

llfix

This repository contains the open-source edition of llfix, a low latency <a href="https://en.wikipedia.org/wiki/Financial_Information_eXchange" target="_blank">FIX protocol</a> engine (MIT licence). A separate commercial edition with additional features and support is available at https://www.llfix.net

The main features of the open-source edition:

• Single-digit microsecond message encoding and decoding (including message serialisation & validations)
• Header-only with no mandatory dependencies (only optional dependency is LibNUMA)
• FIX version agnostic, all versions supported
• C++17
• Linux (GCC & Clang) & Windows (MSVC)
• High determinism : No internal message queueing, no memory allocations on latency-critical paths & vDSO utilisation
• Fast message serialisation via virtual memory
• TCP administration interface, a command-line Python client provided

• Benchmarks
• Benchmarks against FIX8 and QuickFix
• Examples
• Documentation
• Message serialisations & deserialiser tool
• Admin interface
• Configs
• Version history
• Issues and Questions
• References

<a name="benchmarks"></a>

Benchmarks

Server : 2 x Intel Xeon Gold 6134 (2x8 physical cores @3.2 GHz), 256 GB DDR4, RHEL9.4 & GCC 11.4.1

NIC : Solarflare 8000 Series SFN8522 PLUS, 10GBE

Main tunings : Pinning to isolated CPU cores, disabled hyperthreading, maximised CPU frequency, interleaved RAM access

Timestamps : Timestamps in the benchmarks below are recorded from within the application. They are all CPU based (RDTSCP).

All benchmarks can be found in the benchmarks folder. To build them do "make release".

FIXClient TX/send latency benchmark (with Solarflare Onload): Measured latency includes encoding, message serialisation, and enqueueing to NIC (not wire-to-wire) for 1 million messages (benchmarks/networked_client_tx):

Message : 8=FIXT.1.1|9=218|35=D|34=2| 49=CLIENT1|52=20251231-18:21:36.457245600| 56=EXECUTOR|50=SNDR_SUB|57=SRVR_SUB| 11=1|55=NOKIA.HE| 54=1|38=10|44=10000|40=2|59=0| 453=2| 448=PARTY1|447=D|452=1| 448=PARTY2|447=D|452=3| 60=20251231-18:21:36.457245600| 10=221|

| Percentile | Latency | | ------------------------------------- | --------------------------------------------------------------- | | P50 | 573 nanoseconds | | P75 | 798 nanoseconds | | P90 | 1044 nanoseconds | | P95 | 1588 nanoseconds | | P99 | 2663 nanoseconds |

Singlethreaded FIXServer benchmark: 4.7 million messages in total from 32 clients, handled by one thread & on loopback, includes message serialisation & validations (benchmarks/networked_server_rx):

Message : 8=FIXT.1.1|9=188|35=D|34=2| 49=CLIENT1|52=20251231-17:42:03.736004873| 56=EXECUTOR|11=1|55=BMWG.DE| 54=1|38=1|44=5|40=2|59=0| 453=2| 448=PARTY1|447=D|452=1| 448=PARTY2|447=D|452=3| 60=20251231-17:42:03.736004873| 10=077|

| Metric | Measurement | | --------------------- | -------------------------- | | Message throughput | 222423 messages per second | | Latency per message | 4.49 microseconds |

<a name="benchmarks-engines"></a>

Benchmarks against FIX8 and QuickFix

All cross-engine benchmarks were executed on the same hardware and operating system environment and all three engines (llfix, FIX8, and QuickFIX) were built with -O3 flag.

The previously introduced FIXClient TX/send latency benchmark (benchmarks/networked_client_tx) was re-implemented for FIX8 (benchmarks/networked_client_tx_fix8) and Quickfix (benchmarks/networked_client_tx_quickfix).

The full benchmark source code for all three engines can be found in the benchmarks directory. Each benchmark subdirectory contains a README.md file describing the reproduction steps.

The combined results for benchmarks run with Onload are shown below :

• Table 1 compares llfix, Fix8, and QuickFIX with file persistence.
• Table 2 compares llfix with file persistence against Fix8 and QuickFIX with memory persistence.

| Percentile | llfix opensource-edition 1.0.0 (with file persistence) | FIX8 1.0.4 (with file persistence) | Quickfix 17 latency (with file persistence) | | ---------- | -------------------------------------------------------|------------------------------------|-------------------------------------------- | | P50 | 573 nanoseconds | 4189 nanoseconds | 13156 nanoseconds | | P75 | 798 nanoseconds | 4511 nanoseconds | 13520 nanoseconds | | P90 | 1044 nanoseconds | 4986 nanoseconds | 14703 nanoseconds | | P95 | 1588 nanoseconds | 5743 nanoseconds | 15334 nanoseconds | | P99 | 2663 nanoseconds | 12807 nanoseconds | 18595 nanoseconds |

| Percentile | llfix opensource-edition 1.0.0 (with file persistence) | FIX8 1.0.4 (with memory persistence) | Quickfix 17 latency (with memory persistence)| | ---------- | -------------------------------------------------------|--------------------------------------|--------------------------------------------- | | P50 | 573 nanoseconds | 1806 nanoseconds | 7341 nanoseconds | | P75 | 798 nanoseconds | 2162 nanoseconds | 7571 nanoseconds | | P90 | 1044 nanoseconds | 3573 nanoseconds | 7809 nanoseconds | | P95 | 1588 nanoseconds | 5272 nanoseconds | 7893 nanoseconds | | P99 | 2663 nanoseconds | 7416 nanoseconds | 9323 nanoseconds |

<a name="examples"></a>

Examples

Main examples can be found in the examples directory. For a quick start :

git clone https://github.com/CorewareLtd/llfix.git
# Building an running the example FIX server
cd llfix/examples/fix_exchange_simulator
mkdir build
cd build
cmake ..
make
sudo ./example
# And same for fix_trade_client

The following animation shows the FIX client and FIX server examples :

To build with LibNUMA option, specify -DLLFIX_ENABLE_NUMA=ON when running cmake. See engine configs below for NUMA aware allocations.

You can also find other examples under "tests/other_networked_tests" directory :

• "logon_password_authentication" demonstrates specifying logon passwords in FIX clients and validating logon passwords in FIX servers. It also demonstrates setting new logon passwords.
• "raw_data_and_unicode" demonstrates using binary/raw data and unicode characters and sending and handling custom message types.

Repeating groups : In open-source edition, you need to call "specify_repeating_group" for FIX clients and FIX servers. Both examples demonstrate those calls and also encoding/decoding repeating groups.

<a name="documentation"></a>

Documentation

The commercial edition extends the open-source edition and shares the same core. As a result, the commercial documentation also covers the open-source functionality and is available in the following formats:

• Online HTML: https://www.llfix.net/docs/html/
• PDF: https://www.llfix.net/docs/llfix_manual.pdf

<a name="serialisations--deserialiser-tool"></a>

Message serialisations & deserialiser tool

Messages are recorded in a binary format for speed per direction (incoming & outgoing). You should use the deserialiser command line tool in order to view the recorded messages. You can find it in tools directory.

To build it :

• make release on Linux
• use VisualStudio project file on Windows.

CLI options are :

| Option | Description | | --- | --- | | -i <path> | Input serialisation path | | -o <file> | Output file | | -e | Exclude timestamps from output | | -t | Use tag names instead of numbers |

The following animation shows the message deserialiser tool decoding recorded FIX messages:

While specifying an input path, it can contain multiple subfolders. It will be recursively searching for all serialisations and sort them based on serialisation timestamp.

By specifying -t, you can also get textual descriptions instead of tag numbers :

...
TIMESTAMP=19:46:28.832665400
Success=1
BeginString=FIX.4.4|BodyLength=188|MsgType=D|MsgSeqNum=2|SenderCompID=CLIENT1|SendingTime=20251229-19:46:28.82