Karya

Distributed, scalable Task Scheduler built for high throughput.

• How to contribute
• Documentation
• Local Setup
• API Docs

---

Features

1. Schedule delayed job - Will execute after the time duration is passed.

2. Schedule bounded recurring tasks - tasks will execute till specified when to stop.

3. Schedule unbounded recurring tasks - will run indefinitely.

4. Chain Plans - What if a 'task execution' triggers, triggering a new plan? Read more about Chained Plans

Note: This may result in exponentially scheduling tasks. But Karya can be configured to adjust the tree-depth of chained plans.

5. Attach hooks - Hooks can be attached to each job, to perform any reactive action.

6. Trigger multiple Actions in hook - One can attach multiple actions to a plan's hook.

7. Extending Hooks - What if one could trigger a whole new plan instead of taking just reactive actions?

8. Multiple Actions to Configure - Configure from a range of actions on what to do once a task is to be executed.

---

Supported Platforms

Karya Nodes

Karya Nodes can be run completely as docker containers.

• Running Karya Server
• Running Karya Executor
• Running Karya Scheduler

Providers

Karya currently requires the following external components to work: a SQL-based database, a key-value-based one, and a queue.

Karya provides out-of-the-box integration with the below given components with more on the way. For more details, refer - Data Adapters

| Repo Adapter | Locks Adapter | Queue Adapter | |--------------------------------------------------|----------------------------------------------------------|---------------------------------------| | Postgres | Redis | RabbitMQ | | AWS Aurora | ElastiCache | SQS |

Clients

Users can use the following clients to interact with Karya from their preferred environment.

• Java Client
• Python Client
• Javascript Client
• REST API - postman collection

---

Quick Start

This section will help set up Karya locally so that you can start scheduling jobs in no time!

A few things to note: If you haven't already gone through the architecture overview, Karya requires an SQL database, a queue, and a key-value store to work with. For our local setup, we will be using Postgres, Redis, and RabbitMQ, respectively.

For more detailed local setup, please take a look at the local setup documentation.

Pre-requisites

Docker and Docker Compose should be installed on your machine, and the engine is running.

Step 1: Setup providers

Run the below command to set up and run Postgres, Redis, and RabbitMQ containers.

docker-compose -f ./docs/local-setup/providers.docker-compose.yml up -d

Step 2: Create your configuration yml files

Create the following .yml files on your machine:

| File Name | Documentation | Description | Example | |-----------------|----------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------| | providers.yml | Data Adapters | Contains the connection details for Postgres, Redis and RabbitMQ | psql-redis-rabbitmq.providers.yml | | server.yml | Karya Server | Contains the server configuration details. This will serve as the client interfacing side | server.yml | | executor.yml | Karya Executor Connectors | Contains the executor configuration details. This is what will provide the functionality to Karya | executor.yml | | scheduler.yml | Karya Scheduler | Contains the scheduler configuration details. This is the heart of Karya | scheduler.yml |

Step 3: Run Karya Servers

Run the below command to start the Karya servers via docker-compose

NOTE : Make sure to change the paths while mounting the volumes in the below command to the paths where you have stored the .yml files.

docker-compose -f ./docs/local-setup/karya.docker-compose.yml up -d

Step 4: Start using Karya!

Use the client of your choice and start scheduling tasks:

Python Client

Javascript Client

Java Client

Web UI

---

Why Karya?

There are several task schedulers out there. Why to choose Karya? Here are the reasons:

1. High Throughput – Designed for infinite horizontal scaling.
2. Fault-Tolerant – Uses stateless nodes with external data-adapters.
3. Highly Customizable – Configurable with YAML files for data-adapters and connectors.
4. Fast & Safe – Written in Kotlin with structured concurrency using coroutines.