codeQ

Reactive scheduling and completion system backed by persistent queues in KVRocks.

This repository contains the core runtime, HTTP API wiring, and a Helm chart for small clusters. The production service wrapper lives at: https://github.com/codecompany/codeq-service

Why codeQ

codeQ provides:

Persistent queues on KVRocks (Redis protocol).
Pull-based worker claims with leases.
Multi-tenant queue isolation with automatic tenant ID extraction from JWT claims.
NACK + backoff + delayed queues.
DLQ for tasks that exceed maxAttempts.
Result storage and optional callbacks (webhooks).
Worker auth via JWT (JWKS), producer auth via Tikti access tokens (JWKS).
Official SDKs for Java and Node.js/TypeScript with framework integrations.
Distributed tracing with OpenTelemetry support for end-to-end observability.
Optimized performance: O(1) queue operations with pipelined lease repair for low-latency claims even under high load.
Load testing framework: Comprehensive k6 scenarios and Go benchmarks for performance validation and regression testing.

Get started

Local Development with Docker Compose (Recommended)

The quickest way to try codeQ locally:

git clone https://github.com/osvaldoandrade/codeq
cd codeq
docker compose up -d

This starts KVRocks, the codeQ API server, and seeds example tasks. Access at http://localhost:8080.

See Local Development Guide for details on hot reload, testing, and observability.

Install CLI (macOS/Linux/Windows via Git Bash)

curl -fsSL https://raw.githubusercontent.com/osvaldoandrade/codeq/main/install.sh | sh

Requires git and go.

Install CLI via npm (npmjs)

npm i -g @osvaldoandrade/codeq
codeq --help

Upgrade:

npm i -g @osvaldoandrade/codeq@latest

Use SDKs for Java and Node.js

Integrate codeQ into your microservices with official SDKs:

Java (Spring Boot, Quarkus, Micronaut):

<dependency>
    <groupId>io.codeq</groupId>
    <artifactId>codeq-sdk-java</artifactId>
    <version>1.0.0</version>
</dependency>

Node.js/TypeScript (Express, NestJS):

npm install @osvaldoandrade/codeq-client

📚 SDK Documentation:

1) Helm (small cluster)

The chart in this repo deploys codeQ and, by default, a single-node KVRocks instance.

git clone https://github.com/osvaldoandrade/codeq
cd codeq

helm install codeq ./helm/codeq \
  --set secrets.enabled=true \
  --set secrets.webhookHmacSecret=YOUR_SECRET \
  --set config.identityServiceUrl=https://your-auth-server.com \
  --set config.workerJwksUrl=https://your-jwks \
  --set config.workerIssuer=https://issuer

Disable embedded KVRocks and point to your own:

helm install codeq ./helm/codeq \
  --set kvrocks.enabled=false \
  --set config.redisAddr=your-kvrocks:6666

2) Service runtime

The API server and Dockerfile are in the service repo: https://github.com/codecompany/codeq-service

That repo consumes this module and exposes the HTTP API.

Quick API flow

Create a task (producer token):

curl -X POST http://localhost:8080/v1/codeq/tasks \
  -H 'Authorization: Bearer <producer-token>' \
  -H 'Content-Type: application/json' \
  -d '{"command":"GENERATE_MASTER","payload":{"jobId":"j-123"},"priority":3}'

Claim a task (worker JWT):

curl -X POST http://localhost:8080/v1/codeq/tasks/claim \
  -H 'Authorization: Bearer <worker-token>' \
  -H 'Content-Type: application/json' \
  -d '{"commands":["GENERATE_MASTER"],"leaseSeconds":120,"waitSeconds":10}'

Submit result:

curl -X POST http://localhost:8080/v1/codeq/tasks/<id>/result \
  -H 'Authorization: Bearer <worker-token>' \
  -H 'Content-Type: application/json' \
  -d '{"status":"COMPLETED","result":{"ok":true}}'

Specs and docs

Start here: docs/README.md

Key references:

Getting Started Tutorial: docs/00-getting-started.md - Start here for your first experience with codeQ
Overview: docs/01-overview.md - System goals and design principles
Architecture: docs/03-architecture.md - System components and multi-tenant architecture
Security: docs/09-security.md - Authentication, authorization, and tenant isolation
HTTP API: docs/04-http-api.md - Complete API reference
CLI Reference: docs/15-cli-reference.md - CLI command documentation
SDK Integration: sdks/README.md - Official Java and Node.js SDKs
- Java Integration - Spring Boot, Quarkus, Micronaut
- Node.js Integration - Express, NestJS, React
Examples: examples/ - Working examples with Java and Node.js frameworks
Developer Guide: docs/21-developer-guide.md - Contributing and internal architecture
Queue model: docs/05-queueing-model.md - Queue semantics
Storage layout: docs/07-storage-kvrocks.md - KVRocks data structures
Backoff and retries: docs/11-backoff.md - Retry logic
Webhooks: docs/12-webhooks.md - Push notifications
Configuration: docs/14-configuration.md - Config reference
Operations: docs/10-operations.md - Metrics, health checks, and tracing
Workflows: docs/16-workflows.md - GitHub Actions automation
Performance: docs/17-performance-tuning.md - Optimization guide
Load Testing: docs/26-load-testing.md - Load testing framework and benchmarks
Testing: docs/19-testing.md - Test coverage and strategy
Authentication Plugins: docs/20-authentication-plugins.md - Plugin system for custom auth
Persistence Plugins: docs/27-persistence-plugin-system.md - Pluggable storage backends (Redis, Memory, and more)
Design Documents:
- docs/24-queue-sharding-hld.md - Queue sharding high-level design
- docs/25-plugin-architecture-hld.md - Plugin architecture for persistence and auth
Migration: docs/migration.md - Upgrade guide
Contributing: CONTRIBUTING.md - Contribution guidelines

Repo layout

pkg/: public packages (app, config, domain)
internal/: controllers, middleware, services, repositories, providers
helm/codeq: Helm chart
docs/: full specification

License

MIT. See LICENSE.

Codeq

Install / Use

README