FinalMQ - Message exchange framework

Copyright 2020 bexoft GmbH

Author: David Beck, Germany

Contact: david.beck@bexoft.de

C++ Installation - Unix

To build finalmq from source, the following tools and dependencies are needed:

• git
• make
• cmake
• g++
• uuid
• openssl
• nodejs

On Ubuntu/Debian, you can install them with:

sudo apt-get install -y git make cmake g++ uuid-dev libssl-dev nodejs

To get the source, clone the repository of finalmq:

git clone https://github.com/bexoft/finalmq.git

To build and install the C++ finalmq execute the following:

 cd finalmq
 mkdir build
 cd build
 cmake ..
 make
 sudo make install
 sudo ldconfig # refresh shared library cache.

Hint on install location

By default, the package will be installed to /usr/local. However, on many platforms, /usr/local/lib is not part of LD_LIBRARY_PATH. You can add it, but it may be easier to just install to /usr instead. To do this, invoke cmake as follows:

cmake -DCMAKE_INSTALL_PREFIX=/usr ..

If you already built the package with a different prefix, make sure to run "make clean" before building again.

C++ Installation - Windows

To build from source using VC++, follow this instructions:

• Install dependend tools and dependencies listed above

  • Install node.js from https://nodejs.org/en/download/current/

  • Install openSSL You will find a Windows installer at Win32/Win64 OpenSSL Installer for Windows - Shining Light Productions (slproweb.com) Make sure, the environment variable OPENSSL_ROOT_DIR is set to the openssl install directory. Set the cmake variable FINALMQ_USE_SSL=on to compile finalmq with open ssl.

• Open CMakeLists.txt from VisualStudio

• Compile the finalmq project

To build from source using Cygwin or MinGW, follow the Unix installation instructions, above.

Unittests and additional features - Unix

To build the full project of finalmq from source, the additional tools and dependencies are needed:

• gtest/gmock
• protobuf (The unittests test the finalmq-protobuf implementation against the google implementation)
• gcovr (for code coverage)
• doxygen (to generate doxygen out of the code's comments)
• graphviz (to display doxygen diagrams)

On Ubuntu/Debian, you can install them with:

sudo apt-get install -y protobuf-compiler libprotobuf-dev gcovr doxygen graphviz

The gtest/gmock dependency will be automatically resolved by cmake download.

To build the finalmq project with the additional features in debug mode call cmake like this:

cmake -DFINALMQ_BUILD_TESTS=ON -DFINALMQ_BUILD_COVERAGE=ON -DFINALMQ_BUILD_DOXYGEN=ON -DCMAKE_BUILD_TYPE=Debug ..

Now build the project:

make

Afterwards, you can ... ... start tests with:

make verify

... start code coverage statistics:

make coverage

... start doyxgen:

make doc

​

Quick Start

If you want a quick start for finalmq, then look at the examples, "helloworld", "timer", "hl7" and "restapi". You can find the examples in the directory:

examples/cpp

For C#, you can find the examples in examples/csharp

The "helloworld" example demonstrates the asynchronous request/reply calls between client and server. The "timer" example demonstrates events from server to clients. The "restapi" example demonstrates the communication in a REST API manner.

The server application of these examples can be accessed by different interfaces of different technologies.

Please make sure to run only one server at a time, because all examples use the same ports!

You can use telnet for JSON over TCP or the browser input line for JSON over HTTP In the example servers you will find some comments how to access the server commands.

There are also html pages with JavaScript available to demonstrate JSON over HTTP in a script. Just start a server, open a browser and type... ... for helloworld example: localhost:8080/helloworld.html ... for timer example: localhost:8080/timer.htm ... for hl7 example: localhost:8081/hl7.html ... for restapi example: localhost:8080/restapi.html

The according cpp clients can be used to access the servers with Protobuf over TCP.

When you want to use MQTT5, then you have to enable the mqtt5 connections to the broker inside the servers and clients. For mqtt5 be careful of not sending too many requests/events in parallel, because the brokers could drop messages if there are too many messages sent in parallel. This is not a problem of finalmq, but a problem of the configuration of the MQTT broker.

The great thing of finalmq is, that you can use all technologies at the same time!

Architectural Overview

With the architectural overview you will get the knowledge of the layers of finalmq. If this is too early for you, you can skip this chapter and continue with the "Finalmq - Cookbook" chapter.

FinalMQ is a framework for message communication between processes and network nodes. It is based on an asynchronous event loop. This means, events like changing connection state or receiving messages are realized as callbacks into the application. The application has the responsibility, not to sleep or having long running algorithms inside an event callbacks of the framework, because it would affect the timing of other events of other connections. The methods of the framework are thread-safe and can be called from any thread. Typical methods that will be called by the application are e.g. connect() or sendMessage().

The API of FinalMQ has 3 layers. In case the compiler flag FINALMQ_USE_SSL is set, these layers support SSL/TLS encryption, but it exists a dependency to openssl.

The 3 layers of finalMQ:

| Remote Entity | | --------------------- | | Protocol Session | | Stream Connection |

Stream Connection

The first/lowest layer is called Stream Connection. It triggers a callback when new data is ready for read on the socket. This layer is not recognizing begin or end of message. This means it does not care about message framing. When a received() event is triggered, the event handler has to fully read the available data into a buffer, but it is not guaranteed that the received message is complete. The possible kind of sockets are TCP sockets. For unix/linux also Unix Domain Sockets are available. The methods connect() and bind() have an endpoint string that will define the kind of socket, the IP address or hostname and the port or Unix Domain Socket name.

Endpoint examples:

| endpoint | description | | -------------------------- | ------------------------------------------------------------ | | "tcp://localhost:2000" | TCP Socket, hostname: localhost, Port: 2000 | | "tcp://192.168.2.125:3000" | TCP Socket, IP address: 192.168.2.125, Port 3000 | | "tcp://*:2000" | TCP Socket, Wildcard for bind to allow any interface for incoming connections, Port: 2000 | | "ipc://myunixdomain" | Unix Domain Socket with its name |

The main class of this layer is called StreamConnectionContainer. This container manages multi connections. The connection can be incoming connections (bind) and outgoing connections (connect). For one StreamConnectionContainer, it is possible to call multiple times bind() for multiple listening ports (incoming connections) and it is also possible to call multiple times connect() for multiple outgoing connections. The class that represents a connection is called StreamConnection, but the application will only get the interface IStreamConnection as a shared_ptr.

This layer implements SSL/TLS functionalities, in case the compiler-flag FINALMQ_USE_SSL is set.

Protocol Session

The second layer is called Protocol Session. For this layer, an application can implement custom framing protocols as "plugins". When an application receives a message with the received() event, it will deliver always a complete message to the application.

The main class of this layer is called ProtocolSessionContainer. This container manages multi connections. The connection can be incoming connections (bind) and outgoing connections (connect). For one ProtocolSessionContainer, it is possible to call multiple times bind() for multiple listening ports (incoming connections) and it is also possible to call multiple times connect() for multiple outgoing connections. The class that represents a connection is called ProtocolSession, but the application will only get the C++ interface IProtocolSession as a shared_ptr. In this layer, a connection is called session, because there can be protocols implemented which maintain sessions which could live longer than a socket connection. For simple protocols the session will be disconnected as soon the socket is disconnected, but for advanced protocols a session could recognize a socket disconnection, but the session is not disconnected and after a reconnection the session can continue its work. It depends on the protocol, when a session will be disconnected. There could be protocols implemented that guarantee no message lost after reconnection.

Each protocol plugin will have a name that can be used inside the endpoint.

Example:

| endpoints | description | | ----------------------------------- | ------------------------------------------------------------ | | "tcp://localhost:2000:delimiter_lf" | TCP Socket, hostname: localhost, Port: 2000, Framing protocol that looks at a line feed (LF) to separate messages. | | "tcp://*:2000:delimiter_lf" | TCP Socket, Wildcard for any inte