JoSk

"JoSk" is a Node.js task manager for horizontally scaled apps and apps that would need to scale horizontally quickly at some point of growth.

"JoSk" mimics the native API of setTimeout and setInterval and supports CRON expressions. All queued tasks are synced between all running application instances via Redis, MongoDB, or a custom adapter.

The "JoSk" package is made for a variety of horizontally scaled apps, such as clusters, multi-servers, and multi-threaded Node.js instances, that are running either on the same or different machines or even different data centers. "JoSk ensures that the only single execution of each task occurs across all running instances of the application.

"JoSk" is not just for multi-instance apps. It seamlessly integrates with single-instance applications as well, showcasing its versatility and adaptability.

Note: JoSk is the server-only package.

Main features

🏢 Synchronize single task across multiple servers;
🔏 Read locking to avoid simultaneous task executions across complex infrastructure;
📦 Zero dependencies, written from scratch for top performance;
👨‍🔬 ~99% tests coverage;
💪 Bulletproof design, built-in retries, and "zombie" task recovery 🧟🔫.

Prerequisites

redis-server@>=5.0.0 — Redis Server Version (if used with RedisAdapter)
mongod@>=4.0.0 — MongoDB Server Version (if used with MongoAdapter)
node@>=14.20.0 — Node.js version

Older releases compatibility

mongod@<4.0.0 — use josk@=1.1.0
node@<14.20.0 — use josk@=3.0.2
node@<8.9.0 — use josk@=1.1.0

Install:

npm install josk --save

// ES Module Style
import { JoSk, RedisAdapter, MongoAdapter } from 'josk';

// CommonJS
const { JoSk, RedisAdapter, MongoAdapter } = require('josk');

API:

Constructor options for JoSK, MongoAdapter, and RedisAdapter

`new JoSk(opts)`

opts.adapter {RedisAdapter|MongoAdapter} - [Required] Instance of RedisAdapter or MongoAdapter or custom adapter
opts.debug {Boolean} - [Optional] Enable debugging messages, useful during development
opts.autoClear {Boolean} - [Optional] Remove (Clear) obsolete tasks (any tasks which are not found in the instance memory (runtime), but exists in the database). Obsolete tasks may appear in cases when it wasn't cleared from the database on process shutdown, and/or was removed/renamed in the app. Obsolete tasks may appear if multiple app instances running different codebase within the same database, and the task may not exist on one of the instances. Default: false
opts.zombieTime {Number} - [Optional] time in milliseconds, after this time - task will be interpreted as "zombie". This parameter allows to rescue task from "zombie mode" in case when: ready() wasn't called, exception during runtime was thrown, or caused by bad logic. While resetOnInit option helps to make sure tasks are done on startup, zombieTime option helps to solve same issue, but during runtime. Default value is 900000 (15 minutes). It's not recommended to set this value to below 60000 (one minute)
opts.minRevolvingDelay {Number} - [Optional] Minimum revolving delay — the minimum delay between tasks executions in milliseconds. Default: 128
opts.maxRevolvingDelay {Number} - [Optional] Maximum revolving delay — the maximum delay between tasks executions in milliseconds. Default: 768
opts.onError {Function} - [Optional] Informational hook, called instead of throwing exceptions. Default: false. Called with two arguments:
- title {String}
- details {Object}
- details.description {String}
- details.error {Mix}
- details.uid {String} - Internal uid, suitable for .clearInterval() and .clearTimeout()
opts.onExecuted {Function} - [Optional] Informational hook, called when task is finished. Default: false. Called with two arguments:
- uid {String} - uid passed into .setImmediate(), .setTimeout(), or setInterval() methods
- details {Object}
- details.uid {String} - Internal uid, suitable for .clearInterval() and .clearTimeout()
- details.date {Date} - Execution timestamp as JS {Date}
- details.delay {Number} - Execution delay (e.g. interval for .setInterval())
- details.timestamp {Number} - Execution timestamp as unix {Number}

`new RedisAdapter(opts)`

Since v5.0.0

opts.client {RedisClient} - [Required] RedisClient instance, like one returned from await redis.createClient().connect() method
opts.prefix {String} - [Optional] use to create multiple named instances
opts.resetOnInit {Boolean} - [Optional] (use with caution) make sure all old tasks are completed during initialization. Useful for single-instance apps to clean up unfinished that occurred due to intermediate shutdown, reboot, or exception. Default: false

`new MongoAdapter(opts)`

Since v5.0.0

opts.db {Db} - [Required] Mongo's Db instance, like one returned from MongoClient#db() method
opts.prefix {String} - [Optional] use to create multiple named instances
opts.lockCollectionName {String} - [Optional] By default all JoSk instances use the same __JobTasks__.lock collection for locking
opts.resetOnInit {Boolean} - [Optional] (use with caution) make sure all old tasks are completed during initialization. Useful for single-instance apps to clean up unfinished that occurred due to intermediate shutdown, reboot, or exception. Default: false

Initialization

JoSk is storage-agnostic (since v4.0.0). It's shipped with Redis and MongoDB "adapters" out of the box, with option to extend its capabilities by creating and passing a custom adapter

Redis Adapter

JoSk has no dependencies, hence make sure redis NPM package is installed in order to support Redis Storage Adapter. RedisAdapter utilize basic set of commands SET, GET, DEL, EXISTS, HSET, HGETALL, and SCAN. RedisAdapter is compatible with all Redis-alike databases, and was well-tested with Redis and KeyDB

import { JoSk, RedisAdapter } from 'josk';
import { createClient } from 'redis';

const redisClient = await createClient({
  url: 'redis://127.0.0.1:6379'
}).connect();

const jobs = new JoSk({
  adapter: new RedisAdapter({
    client: redisClient,
    prefix: 'app-scheduler',
  }),
  onError(reason, details) {
    // Use onError hook to catch runtime exceptions
    // thrown inside scheduled tasks
    console.log(reason, details.error);
  }
});

MongoDB Adapter

JoSk has no dependencies, hence make sure mongodb NPM package is installed in order to support MongoDB Storage Adapter. Note: this package will add two new MongoDB collections per each new JoSk(). One collection for tasks and second for "Read

Josk

Install / Use

README