Runqy

<p align="center"> <img src="assets/logo.svg" alt="runqy logo" width="80" height="80"> </p> <h1 align="center">runqy</h1> <p align="center"> <strong>Open-source task queue for AI workloads. Deploy workers anywhere, from your laptop to the cloud.</strong> </p> <p align="center"> <a href="https://github.com/Publikey/runqy/stargazers"><img src="https://img.shields.io/github/stars/Publikey/runqy?style=flat&logo=github" alt="GitHub Stars"></a> <a href="https://github.com/Publikey/runqy/blob/main/LICENSE"><img src="https://img.shields.io/github/license/Publikey/runqy" alt="License"></a> <a href="https://golang.org/"><img src="https://img.shields.io/badge/Go-1.21+-00ADD8?style=flat&logo=go" alt="Go Version"></a> <a href="https://github.com/Publikey/runqy-python"><img src="https://img.shields.io/badge/Python_SDK-Available-3776AB?style=flat&logo=python" alt="Python SDK"></a> <a href="https://github.com/Publikey/runqy/actions"><img src="https://img.shields.io/github/actions/workflow/status/Publikey/runqy/release.yml" alt="Build Status"></a> </p> <p align="center"> <a href="https://docs.runqy.com"><strong>Documentation</strong></a> · <a href="https://runqy.com"><strong>Website</strong></a> · <a href="#examples">Examples</a> · <a href="CONTRIBUTING.md">Contributing</a> </p>

---

<p align="center"> <img src="assets/demo.gif" alt="Runqy demo — from zero to task result in 90 seconds" width="800"> </p>

---

Why Runqy?

🌍 Workers run anywhere — Your laptop, on-prem servers, AWS, Azure, Runpod, any machine with an internet connection. Learn more →
🚀 Zero-touch deployment — Workers pull code from Git, install dependencies, and start processing automatically. No manual setup. Learn more →
📄 Simple YAML config — Define a queue in a few lines. One YAML file, one queue. Learn more →
🔐 Built-in secrets — Pass secrets to workers via encrypted env vars. Learn more →
🐍 Go server + Python SDK — Robust Go server, familiar Python developer experience. Learn more →
📊 Web monitoring UI — Real-time dashboard with Prometheus metrics. Learn more →

Feature Comparison

| Feature | Runqy | Celery | Temporal | Modal | BullMQ | Inngest | |---------|-------|--------|----------|-------|--------|---------| | Self-hosted | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | | Workers anywhere | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | | Auto-deploy from Git | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | | Deployment YAML | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | | Built-in secrets | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | | Monitoring UI | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |

---

Quick Start

Get Runqy running in under 60 seconds:

# 1. Start the stack
curl -O https://raw.githubusercontent.com/Publikey/runqy/main/docker-compose.quickstart.yml
docker-compose -f docker-compose.quickstart.yml up -d

# 2. Enqueue a task
pip install runqy-python
python -c "
from runqy_python import RunqyClient
client = RunqyClient('http://localhost:3000', api_key='dev-api-key')
task = client.enqueue('quickstart-oneshot', {'message': 'Hello World!'})
print(f'Task ID: {task.task_id}')
"

# 3. Check results
open http://localhost:3000/monitoring/

See the Quickstart Guide for the full walkthrough.

Define a Queue

A queue is a simple YAML file:

queues:
  image-resize:
    priority: 5
    deployment:
      # Worker code: https://github.com/acme/image-worker
      git_url: "https://github.com/acme/image-worker.git"
      branch: "main"
      startup_cmd: "python main.py"
      mode: "one_shot"

Deploy it:

runqy config create -f queue.yaml

See the Queue Configuration Reference for all options.

Write a Task

from runqy import task, load

@load
def setup():
    """Load models once when worker starts"""
    import torch
    return torch.load('my_model.pt')

@task
def process_image(image_url: str, model) -> dict:
    """Runs on every task execution"""
    result = model.predict(image_url)
    return {"prediction": result, "confidence": 0.95}

See the Python SDK Reference for the full API.

Enqueue Tasks

Three ways to enqueue:

# CLI
runqy task enqueue -q image-resize -p '{"image":"img001.jpg","size":256}'

# REST API
curl -s POST localhost:3000/queue/add \
  -H "X-API-Key: dev-api-key" \
  -d '{"queue":"image-resize","data":{"image":"img002.jpg"}}'

# Python SDK
from runqy_python import RunqyClient
client = RunqyClient('http://localhost:3000', api_key='dev-api-key')
task = client.enqueue('image-resize', {'image': 'img003.jpg'})

See the API Reference for all endpoints.

---

Examples

Explore real-world use cases:

• quickstart-oneshot — Simple task execution
• quickstart-longrunning — Long-running worker processes
• data-pipeline — Multi-step data processing (API calls, ETL)
• webhook-processor — Event-driven webhook handling (Stripe, GitHub)
• scheduled-tasks — Cron-like healthchecks, reports, and cleanup
• multi-queue — Priority-based routing (critical, standard, bulk)
• gpu-inference — GPU-accelerated image generation with Stable Diffusion
• star-runqy — Vault secrets management tutorial

---

Installation

Quick Install

Linux/macOS:

curl -fsSL https://raw.githubusercontent.com/publikey/runqy/main/install.sh | sh

Windows (PowerShell):

iwr https://raw.githubusercontent.com/publikey/runqy/main/install.ps1 -useb | iex

Docker

docker pull ghcr.io/publikey/runqy:latest

From Source

git clone https://github.com/Publikey/runqy.git
cd runqy
go build -o runqy ./app

See the Installation Guide for detailed instructions.

Requirements

• Redis + PostgreSQL

Server Configuration

Configure the server via environment variables:

export REDIS_HOST=localhost:6379
export RUNQY_API_KEY=your-secret-key

See the Configuration Reference for all options.

CLI Reference

Manage your deployment locally or remotely:

runqy queue list                    # List all queues
runqy config create -f queue.yaml   # Deploy a queue

runqy task enqueue -q myqueue -p '{"key":"value"}'  # Enqueue task
runqy task list myqueue                              # List tasks
runqy task get myqueue <task_id>                     # Get task result

runqy worker list                   # List active workers

See the CLI Reference for all commands.

Monitoring

Access the built-in web dashboard at /monitoring:

<p align="center"> <img src="docs/images/monitoring-dashboard.png" alt="Runqy Dashboard" width="700"> </p> <details> <summary>📊 More screenshots</summary>

Queue Overview — Status, pending/active/completed counts, latency per queue:

<p align="center"> <img src="docs/images/monitoring-queues.png" alt="Runqy Queues" width="700"> </p>

Workers — CPU/RAM usage, assigned queues, heartbeat status:

<p align="center"> <img src="docs/images/monitoring-workers.png" alt="Runqy Workers" width="700"> </p> </details>

Runqy also exposes Prometheus metrics at /metrics. See the Monitoring Guide for Grafana dashboards and alerting.

Architecture

Tasks flow from clients → runqy server → queues → workers running anywhere. Workers are stateless and pull code from Git on startup.

<p align="center"> <img src="assets/architecture.png" alt="runqy architecture" width="700"> </p>

Zero-touch Deployment: Workers connect to the server, pull your code from Git, install dependencies, and start processing — no manual setup required.

<p align="center"> <img src="assets/code_pull.png" alt="zero-touch deployment" width="700"> </p>

Links

• 📖 Documentation — Complete guides and API reference
• 🌐 Website — Project homepage
• 🐍 Python SDK — Client library
• 🔧 Worker Runtime — Task processor
• 🤝 Contributing — How to contribute
• 📄 License — MIT License

---

<p align="center"> <strong>Your workers, your machines, your rules.</strong><br> Built on <a href="https://github.com/hibiken/asynq">asynq</a> • Made with ❤️ for AI developers </p>