remoulade

A fast and reliable distributed task processing library for Python 3. Fork of dramatiq.io

<hr/>

Changelog: https://remoulade.readthedocs.io/en/latest/changelog.html <br/> Documentation: https://remoulade.readthedocs.io

<hr/>

Installation

If you want to use it with RabbitMQ

    uv pip install 'remoulade[rabbitmq]'

or if you want to use it with Redis

   uv pip install 'remoulade[redis]'

Quickstart

Make sure you've got RabbitMQ running, then create a new file called example.py:

from remoulade.brokers.rabbitmq import RabbitmqBroker
import remoulade
import requests
import sys

broker = RabbitmqBroker()
remoulade.set_broker(broker)


@remoulade.actor
def count_words(url):
    response = requests.get(url)
    count = len(response.text.split(" "))
    print(f"There are {count} words at {url!r}.")


broker.declare_actor(count_words)

if __name__ == "__main__":
    count_words.send(sys.argv[1])

In one terminal, run your workers:

   remoulade example

In another, start enqueueing messages:

   python3 example.py http://example.com
   python3 example.py https://github.com
   python3 example.py https://news.ycombinator.com

Visit the user guide to see more features!.

Releasing

If you want to contribute to the project. First make a Pull request and get approval from a Maintainer and then:

Merge your pull request
Push a tag (must start with a v) from the commit you want (usually, the master branch) a. You can create a new release on Github (recommended). This will create and push the tag for you. b. Update your local clone, checkout master, git tag and then git push --tags

This will trigger a CI/CD pipeline that publish the package

Dashboard

Check out SuperBowl a dashboard for real-time monitoring and administrating all your Remoulade tasks. See the current progress, enqueue, requeue, cancel and more ... Super easy to use !.

Kubernetes

Remoulade is tailored to run transparently in containers on Kubernetes and to make the most of their features. This does not mean it cannot run outside of Kubernetes ;)

Monitoring/Prometheus

Remoulade provides a Prometheus Middleware that exposes various metrics (processed messages, retries, fails etc.), Check it out.

Tracing/OpenTelemtry

Take a look at opentelemetry-instrumentation-remoulade if you want to enable tracing for Remoulade operations (enqueue, message processing, retries etc.). See here for an example. The library follows OpenTelemetry standards.

License

remoulade is licensed under the LGPL. Please see COPYING and COPYING.LESSER for licensing details.

Remoulade

Install / Use

README