Chronicle Queue

= Chronicle Queue Peter Lawrey, Rob Austin :css-signature: demo :toc: macro :toclevels: 2 :icons: font :source-highlighter: rouge

Chronicle Queue is a broker-less, off-heap Java library for ultra-low-latency, persisted messaging at millions of events/sec.

image:https://maven-badges.herokuapp.com/maven-central/net.openhft/chronicle-queue/badge.svg[caption="",link=https://maven-badges.herokuapp.com/maven-central/net.openhft/chronicle-queue] image:https://javadoc.io/badge2/net.openhft/chronicle-queue/javadoc.svg[link="https://www.javadoc.io/doc/net.openhft/chronicle-queue/latest/index.html"] image:https://img.shields.io/github/license/OpenHFT/Chronicle-Queue[GitHub] image:https://img.shields.io/gitter/room/OpenHFT/Lobby.svg?style=popout[link="https://gitter.im/OpenHFT/Lobby"] image:https://img.shields.io/badge/release%20notes-subscribe-brightgreen[link="https://chronicle.software/release-notes/"] image:https://sonarcloud.io/api/project_badges/measure?project=OpenHFT_Chronicle-Queue&metric=alert_status[link="https://sonarcloud.io/dashboard?id=OpenHFT_Chronicle-Queue"]

image::docs/images/Queue_line.png[width=20%]

toc::[]

== Overview

Chronicle Queue is a persisted low-latency messaging framework for high performance applications. It supports multiple writers to a queue via locking, and multiple lock-less concurrent readers of the queue.

This project covers the Java version of Chronicle Queue. A {cpp} version of this project is also available and supports Java/{cpp} interoperability plus additional language bindings e.g. Python. If you are interested in evaluating the {cpp} version please contact mailto:sales@chronicle.software[sales@chronicle.software].

At first glance Chronicle Queue can be seen as simply another queue implementation. However, it has major design choices that should be emphasised. Using off-heap storage, Chronicle Queue provides an environment where applications do not suffer from Garbage Collection (GC). When implementing high-performance and memory-intensive applications (you heard the fancy term "bigdata"?) in Java, one of the biggest problems is garbage collection.

Chronicle Queue allows messages to be added to the end of a queue ("appended"), read from the queue ("tailed"), and also supports random-access seek.

link:https://player.vimeo.com/video/201989439[Why Use Chronicle Queue Between Microservices?]

A number of relevant system properties are listed in link:docs/systemProperties.adoc[systemProperties.adoc].

== What Is Chronicle Queue?

You could consider a Chronicle Queue to be similar to a low latency broker-less durable/persisted topic that can contain messages of different types and sizes. Chronicle Queue is a distributed unbounded persisted queue that:

• supports asynchronous RMI and Publish/Subscribe interfaces with microsecond latencies.
• passes messages between JVMs in under a microsecond
• passes messages between JVMs on different machines via replication in under 10 microseconds (<<Chronicle Queue Enterprise Edition,Enterprise feature>>)
• provides stable, soft real-time latencies into the millions of messages per second for a single thread to one queue; with total ordering of every event.

When publishing 40-byte messages, a high percentage of the time we achieve latencies under 1 microsecond. The 99th percentile latency is the worst 1 in 100, and the 99.9th percentile is the worst 1 in 1000 latency.

.Latency to send/receive on the same machine. [width="60%",options="header"] |======= | Batch Size | 10 million events per minute | 60 million events per minute | 100 million events per minute | 99%ile | 0.78 µs | 0.78 µs | 1.2 µs | 99.9%ile | 1.2 µs | 1.3 µs | 1.5 µs |=======

.Latency to send/receive on a second machine. [width="60%",options="header"] |======= | Batch Size | 10 million events per minute | 60 million events per minute | 100 million events per minute | 99%ile | 20 µs | 28 µs | 176 µs | 99.9%ile | 901 µs | 705 µs | 5,370 µs |=======

NOTE: 100 million events per minute is sending an event every 660 nanoseconds; replicated and persisted.

IMPORTANT: This performance is not achieved using a large cluster of machines. This is using one thread to publish, and one thread to consume.

=== Design Motivation and Features

Chronicle Queue is designed to:

• be a "record everything store" which can read with microsecond real-time latency. This supports even the most demanding High Frequency Trading systems. However, it can be used in any application where the recording of information is a concern.

• support reliable replication with notification to either the appender (writer of message) or a tailer (reader of message), when a message has been successfully replicated.

==== Persistence

Chronicle Queue assumes disk space is cheap compared with memory. Chronicle Queue makes full use of the disk space you have, and so you are not limited by the main memory of your machine. If you use spinning HDD, you can store many TBs of disk space for little cost.

The only extra software that Chronicle Queue needs to run is the operating system. It doesn't have a broker; instead it uses your operating system to do all the work. If your application dies, the operating system keeps running for seconds longer, so no data is lost; even without replication.

As Chronicle Queue stores all saved data in memory-mapped files, this has a trivial on-heap overhead, even if you have over 100 TB of data.

==== Efficiency

Chronicle put significant effort into achieving very low latency. In other products which focus on support of web applications, latencies of less than 40 milliseconds are fine as they are faster than you can see; for example, the frame rate of cinema is 24 Hz, or about 40 ms.

Chronicle Queue aims to achieve latencies of under 40 microseconds for 99% to 99.99% of the time. Using Chronicle Queue without replication, we support applications with latencies below 40 microseconds end-to-end across multiple services. Often the 99% latency of Chronicle Queue is entirely dependent on the choice of operating system and hard disk sub-system.

==== Compression

Replication for Chronicle Queue supports Chronicle Wire Enterprise. This supports a real-time compression which calculates the deltas for individual objects, as they are written. This can reduce the size of messages by a factor of 10, or better, without the need for batching; that is, without introducing significant latency.

Chronicle Queue also supports LZW, Snappy, and GZIP compression. These formats however add significant latency. These are only useful if you have strict limitations on network bandwidth.

==== Delivery mode semantics

Chronicle Queue supports a number of semantics:

• Every message is replayed on restart.
• Only new messages are played on restart.
• Restart from any known point using the index of the entry.
• Replay only the messages you have missed. This is supported directly using the methodReader/methodWriter builders.

==== Using high resolution timings across machines

On most systems System.nanoTime() is roughly the number of nanoseconds since the system last rebooted (although different JVMs may behave differently). This is the same across JVMs on the same machine, but wildly different between machines. The absolute difference when it comes to machines is meaningless. However, the information can be used to detect outliers; you can't determine what the best latency is, but you can determine how far off the best latencies you are. This is useful if you are focusing on the 99th percentile latencies. We have a class called RunningMinimum to obtain timings from different machines, while compensating for a drift in the nanoTime between machines. The more often you take measurements, the more accurate this running minimum is.

==== Compacting logs

Chronicle Queue manages storage by cycle. You can add a StoreFileListener which will notify you when a file is added, and when it is no longer retained. You can move, compress, or delete all the messages for a day, at once. NOTE : Unfortunately on Windows, if an IO operation is interrupted, it can close the underlying FileChannel.

==== Avoid Interrupts

Due to performance reasons, we have removed checking for interrupts in the chronicle queue code. Because of this, we recommend that you avoid using chronicle queue with code that generates interrupts. If you can not avoid generating interrupts then we suggest that you create a separate instance of Chronicle Queue per thread.

=== Usage

Chronicle Queue is most often used for producer-centric systems where you need to retain a lot of data for days or years. For statistics see https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vTe-ijX-uRMc86pB1r-qPUIDZmzI0drPQtvUiGiU8p6WEq98HHDO47HXfV_dk_q6Tmhr1fq2pLxLkqv/pubhtml[Usage of Chronicle-Queue]

IMPORTANT: Chronicle Queue does not support operating off any network file system, be it NFS, AFS, SAN-based storage or anything else. The reason for this is those file systems do not provide all the required primitives for memory-mapped files Chronicle Queue uses. If any networking is needed (e.g. to make the data accessible to multiple hosts), the only supported way is Chronicle Queue Replication (Enterprise feature).

==== What is a producer-centric system?

Most messaging systems are consumer-centric. Flow control is implemented to avoid the consumer ever getting overloaded; even momentarily. A common example is a server supporting multiple GUI users. Those users might be on different machines (OS and hardware), different qualities of network (latency and bandwidth), doing a variety of other things at different times. For this reason it makes sense for the client consumer to tell the producer when to back off, delaying any data until the consumer is ready to take more data.

Chronicle Queue is a producer-centric solution and does everything possible to never push back on the producer,