This project is no longer actively supported. It is made available as read-only.

CMB (Cloud Message Bus) README

A highly available, horizontally scalable queuing and notification service compatible with AWS SQS and SNS. This document covers these topics:

FAQ
User Forum
Binaries for Download
Accessing CQS/CNS Using AWS SDK
Brief Tutorial
Quickstart Guide
Monitoring, Logging
Multi Datacenter Support and Failover
Known Limitations

If you have trouble installing or using CMB check our FAQ first.

User Forum

If you have any questions or comments please go to our user forum at

https://groups.google.com/forum/#!forum/cmb-user-forum

Binaries for Download

If you do not want to build from source you can download binary distributions from the release section of this github repository.

Accessing CQS/CNS Using AWS SDK

If you prefer getting started by looking at some code, here's a complete end-to-end client-side example for accessing CQS/CNS services using the AWS SDK:

https://github.com/Comcast/cmb/blob/master/examples/java/Example.java

In this example, we first create a queue and a topic. Next we subscribe the queue and also an http endpoint to the topic and finally we publish, receive and delete messages. Confirmation of pending subscriptions is also demonstrated.

Brief Tutorial

CMB consists of two separate services, CQS and CNS. CQS offers queuing services while CNS offers publish / subscribe notification services. Both services are API-compatible with Amazon Web Services SNS (Simple Notification Service) and SQS (Simple Queuing Service). CNS currently supports these protocols for subscribers: HTTP, CQS, SQS and email. CMB services are implemented with a Cassandra / Redis backend and are designed for high availability and horizontal scalability.

Note: While CQS and CNS are two functionally distinct services, CNS uses CQS internally to distribute the fan-out work. Therefore, CNS cannot be deployed without CQS, however, you can deploy CQS in isolation if you do not need the pub-sub capabilities of CNS.

The most basic CMB system consists of one of each

CQS Service Endpoint (HTTP endpoint for CQS web service)
CNS Service Endpoint (HTTP endpoint for CNS web service)
CNS Publish Worker (required by CNS, used to publish messages to endpoints)
Cassandra Ring (persistence layer, used by CNS and CQS)
Sharded Redis (caching layer, used by CQS)

For testing purposes all five components can be run on a single host in a single JVM but a production deployment typically uses separate hosts for each.

For a detailed documentation of the CNS / CQS APIs please refer to the Amazon SNS / SQS specifications here:

http://docs.amazonwebservices.com/sns/latest/api/Welcome.html http://docs.amazonwebservices.com/AWSSimpleQueueService/latest/APIReference/Welcome.html

Accessing CNS / CQS:

There are three different ways to access CNS / CQS services:

Using the web based CMB Admin UI:

The Admin UI is a simple Web UI for testing and administration purposes. To access the CMB Admin UI use any web browser and go to

Web UI URL: http://<cqs_host>:6059/webui/

Using the AWS SDK for Java or similar language bindings:

Amazon offers a Java SDK to access SNS / SQS and other AWS services. Since CNS / CQS are API-compatible you can use the AWS SDK to access our implementation in the same way. Thus, instead of having to engineer your own REST requests you can get started with a few lines of simple code as the following example illustrates.

BasicAWSCredentials credentialsUser = new BasicAWSCredentials("<access_key>", "<secret_key>");

String cqsServerUrl = "http://<cqs_host>:<cqs_port>/";

AmazonSQSClient sqs = new AmazonSQSClient(credentialsUser); sqs.setEndpoint(cqsServerUrl);

Random randomGenerator = new Random(); String queueName = QUEUE_PREFIX + randomGenerator.nextLong();

HashMap<String, String> attributeParams = new HashMap<String, String>(); CreateQueueRequest createQueueRequest = new CreateQueueRequest(queueName); createQueueRequest.setAttributes(attributeParams); String queueUrl = sqs.createQueue(createQueueRequest).getQueueUrl();

Amazon offers a few other language bindings of its SDK and there are also a number of third party SDKs available for languages not supported by Amazon.

Sending REST requests directly to the service endpoint:

All CNS / CQS features can also be accessed by sending REST requests as HTTP GET or POST directly to the service endpoints. Note that you need to timestamp and digitally sign every request if signature verification is enabled (signature verification is disabled by default, you can enable this feature by changing cmb.properties).

Example REST request to create a CQS queue using curl:

curl -d "Action=CreateQueue&SignatureMethod=HmacSHA256&AWSAccessKeyId=48JT2LKD3TX9X5JD6NMM&QueueName=TSTQ_-3098387640939725337&SignatureVersion=2&Version=2011-10-01&Signature=FemvuycfOczDIySdw9K4fjHvBWDm9W4iLDFUNQK220M%3D&Timestamp=2012-08-04T00%3A14%3A54.157Z" http://<cqs_host>:<cqs_port>

Example response:

Quickstart Guide

CMB comes with an embedded Jetty server. As a result the required components (CQS Service Endpoint, CNS Service Endpoint and CNS Publish Worker) can all be conveniently launched using the cmb.sh script within a single JVM. The only external components you need to install separately are Cassandra and Redis (make sure Redis does not persist to disk!).

Build CMB from source

git clone https://github.com/Comcast/cmb.git mvn -Dmaven.test.skip=true assembly:assembly

or download binary from

http://cmb-releases.s3-website-us-west-1.amazonaws.com/
Unpack binary from target folder:

tar -xzvf cmb-distribution-<version>.tar.gz
Edit cmb.properties with a particular focus on the Redis and Cassandra settings. For a single standalone CMB node ensure the CNS and CQS options are fully enabled (default).

cmb.cns.serviceEnabled=true cmb.cqs.serviceEnabled=true cmb.cns.publisherEnabled=true cmb.cns.publisherMode=Consumer,Producer
Install the correct schema version in your Cassandra ring

NOTE: CMB does not work with Cassandra versions prior to 1.0.10. For Cassandra versions 1.1.X and 1.2.X edit cassandra.yaml to activate the global row cache:

row_cache_size_in_mb: 100

To install the schema, use the appropriate command based on your Cassandra version

Using cli (deprecated):

/<path_to_cassandra>/bin/cassandra-cli -h localhost -f cmb/schema/cassandra_1.0.schema

/<path_to_cassandra>/bin/cassandra-cli -h localhost -f cmb/schema/cassandra_1.1.schema

/<path_to_cassandra>/bin/cassandra-cli -h localhost -f cmb/schema/cassandra_1.2.schema

Using cql:

/<path_to_cassandra>/bin/cqlsh localhost 9160 -f schema/cassandra_1.2.cql.schema

NOTE: In newer versions of Cassandra thrift is disabled by default, as a result CMB cannot connect to Cassandra. A solution to this is to run "nodetool enablethrift" while Cassandra is running.

NOTE: If you want the keyspaces to be replicated, you will need to change the keyspace definitions before adding the schema. Make sure to pick a suitable replication factor for your Cassandra ring, for example, RF=1 for single node rings and RF=3 for 3 nodes or larger rings.
Redis version 2.6 or higher is required. Also be sure Redis does not persist to disk.

Edit the redis.conf file and disable all persistence by commenting out the three lines starting with "save".

save 900 1

save 300 10

save 60 10000
Start your CMB node:

./bin/cmb.sh
Check if web UI is available at localhost:6059/webui/ (login with username cns_internal and password cns_internal).

Monitoring, Logging

By default all CMB components (CNS and CQS API Servers, CNS Worker Nodes) write INFO level logging into a file /tmp/cmb.log. You can adjust the logging behavior by editing config/log4.p

Cmb

Install / Use

README

save 900 1

save 300 10

save 60 10000