tl;dr

in-memory message queue system
runs stand-alone, via Docker or embedded
Amazon SQS-compatible interface
fully asynchronous implementation, no blocking calls
optional UI, queue persistence
created and maintained by:

Summary

ElasticMQ is a message queue system, offering an actor-based Scala and an SQS-compatible REST (query) interface.

ElasticMQ follows the semantics of SQS. Messages are received by polling the queue. When a message is received, it is blocked for a specified amount of time (the visibility timeout). If the message isn't deleted during that time, it will be again available for delivery. Moreover, queues and messages can be configured to always deliver messages with a delay.

The focus in SQS (and ElasticMQ) is to make sure that the messages are delivered. It may happen, however, that a message is delivered twice (if, for example, a client dies after receiving a message and processing it, but before deleting). That's why clients of ElasticMQ (and Amazon SQS) should be idempotent.

As ElasticMQ implements a subset of the SQS query (REST) interface, it is a great SQS alternative both for testing purposes (ElasticMQ is easily embeddable) and for creating systems which work both within and outside of the Amazon infrastructure.

A simple UI is available for viewing real-time queue statistics.

Community

Installation: stand-alone

You can download the stand-alone distribution here:

wget https://s3-eu-west-1.amazonaws.com/softwaremill-public/elasticmq-server-$VERSION.jar

Java 8 or above is required for running the server.

Simply run the jar and you should get a working server, which binds to localhost:9324:

java -jar elasticmq-server-$VERSION.jar

ElasticMQ uses Typesafe Config for configuration. To specify custom configuration values, create a file (e.g. custom.conf), fill it in with the desired values, and pass it to the server:

java -Dconfig.file=custom.conf -jar elasticmq-server-$VERSION.jar

The config file may contain any configuration for Akka and ElasticMQ. Current ElasticMQ configuration values are:

include classpath("application.conf")

# What is the outside visible address of this ElasticMQ node
# Used to create the queue URL (may be different from bind address!)
node-address {
  protocol = http
  host = localhost
  port = 9324
  context-path = ""
}

rest-sqs {
  enabled = true
  bind-port = 9324
  bind-hostname = "0.0.0.0"
  # Possible values: relaxed, strict
  sqs-limits = strict
}

rest-stats {
  enabled = true
  bind-port = 9325
  bind-hostname = "0.0.0.0"
}

# Should the node-address be generated from the bind port/hostname
# Set this to true e.g. when assigning port automatically by using port 0.
generate-node-address = false

queues {
  # See next sections
}

queues-storage {
  # See next sections
}

# Region and accountId which will be included in resource ids
aws {
  region = us-west-2
  accountId = 000000000000
}

You can also provide an alternative Logback configuration file (the default is configured to log INFO logs and above to the console):

java -Dlogback.configurationFile=my_logback.xml -jar elasticmq-server-$VERSION.jar

How are queue URLs created

Some of the responses include a queue URL. By default, the URLs will use http://localhost:9324 as the base URL. To customize, you should properly set the protocol/host/port/context in the node-address setting (see above).

You can also set node-address.host to a special value, "*", which will cause any queue URLs created during a request to use the path of the incoming request. This might be useful e.g. in containerized (Docker) deployments.

Note that changing the bind-port and bind-hostname settings do not affect the queue URLs in any way unless generate-node-address is true. In that case, the bind host/port are used to create the node address. This is useful when the port should be automatically assigned (use port 0 in such case, the selected port will be visible in the logs).

Automatically creating queues on startup

Queues can be automatically created on startup by providing appropriate configuration:

The queues are specified in a custom configuration file. For example, create a custom.conf file with the following:

# the include should be done only once, at the beginning of the custom configuration file
include classpath("application.conf")

queues {
  queue1 {
    defaultVisibilityTimeout = 10 seconds
    delay = 5 seconds
    receiveMessageWait = 0 seconds
    deadLettersQueue {
      name = "queue1-dead-letters"
      maxReceiveCount = 3 // from 1 to 1000
    }
    fifo = false
    contentBasedDeduplication = false
    copyTo = "audit-queue-name"
    moveTo = "redirect-queue-name"
    tags {
      tag1 = "tagged1"
      tag2 = "tagged2"
    }
  }
  queue1-dead-letters { }
  audit-queue-name { }
  redirect-queue-name { }
}

All attributes are optional (except name and maxReceiveCount when a deadLettersQueue is defined). copyTo and moveTo attributes allow to achieve behavior that might be useful primarily for integration testing scenarios - all messages could be either duplicated (using copyTo attribute) or redirected (using moveTo attribute) to another queue.

FIFO queue creation

To create FIFO queue set value of fifo config parameter to true. You can add .fifo suffix to queue name yourself (a name containing . has to be surrounded with quotes), for example:

queues {
  "testQueue.fifo" {
    fifo = true
    contentBasedDeduplication = true
  }
}

If not then suffix will be added automatically during queue creation.

Persisting queues configuration

Queues configuration can be persisted in an external config file in the HOCON format. Note that only the queue metadata (which queues are created, and with what attributes) will be stored, without any messages.

To enable the feature, create a custom configuration file with the following content:

# the include should be done only once, at the beginning of the custom configuration file
include classpath("application.conf")

queues-storage {
  enabled = true
  path = "/path/to/storage/queues.conf"
}

Any time a queue is created, deleted, or its metadata change, the given file will be updated.

On startup, any queues defined in the given file will be created. Note that the persisted queues configuration takes precedence over queues defined in the main configuration file (as described in the previous section) in the queues section.

Persisting queues and messages to SQL database

Queues and their messages can be persisted to SQL database in runtime. All events like queue or message creation, deletion or update will be stored in H2 in-file database, so that the entire ElasticMQ state can be restored after server restart.

To enable the feature, create a custom configuration file with the following content:

# the include should be done only once, at the beginning of the custom configuration file
include classpath("application.conf")

messages-storage {
  enabled = true
}

By default, the database file is stored in /data/elasticmq.db. In order to change it, custom JDBC uri needs to be provided:

# the include should be done only once, at the beginning of the custom configuration file
include classpath("application.conf")

messages-storage {
  enabled = true
  uri = "jdbc:h2:/home/me/elasticmq"
}

On startup, any queues and their messages persisted in the database will be recreated. Note that the persisted queues take precedence over the queues defined in the main configuration file (as described in the previous section) in the queues section.

ElasticMQ dependencies in SBT

// Scala 2.13 and 2.12
val elasticmqSqs        = "org.elasticmq" %% "elasticmq-rest-sqs" % Version

If you don't want the SQS interface, but just use the actors directly, you can add a dependency only to the core module:

val elasticmqCore       = "org.elasticmq" %% "elasticmq-core" % Version

If you want to use a snapshot version, you will need to add the https://oss.sonatype.org/content/repositories/snapshots/ repository to your configuration.

ElasticMQ dependencies in Maven

Dependencies:

<dependency>
    <groupId>org.elasticmq</groupId>
    <artifactId>elasticmq-rest-sqs_2.12</artifactId>
    <version>${version}</version>
</dependency>