<img src="/images/logo.png" alt="cTRL Logo"/>

Документация на русском: https://github.com/eltaline/ctrl/blob/master/README-RUS.md

cTRL is a server written in Go language that uses a <a href=https://github.com/eltaline/nutsdb>modified</a> version of the NutsDB database, as a backend for a continuous queue of tasks and saving the result of executing commands from given tasks in command interpreters like /bin/bash on servers where this service will be used. Using cTRL, you can receive tasks via the HTTP protocol with commands for executing them on the server and limit the number of simultaneously executed tasks.

Current stable version: 1.1.11

• <a href=/CHANGELOG.md>Changelog</a>

Features

• Multi threading
• Supports HTTPS and Auth/IP authorization
• Supported HTTP methods: GET, POST
• Parallel execution of tasks in realtime
• Receiving tasks in the queue for deferred parallel execution of commands
• Limiting the maximum number of threads per virtual host
• Supports continuous task execution queue
• Automatic simultaneous execution of only one identical task
• Supported interpreters: /bin/bash, /bin/sh
• Supported formats: JSON

Requirements

• Operating Systems: Linux, BSD, Solaris, OSX
• Architectures: amd64, arm64, ppc64 and mips64, with only amd64 tested

Real application

We use this server to perform delayed tasks for parallel video conversion using ffmpeg with limiting processes on several video servers with Nvidia GPU, but the server can execute any commands through the shell interpreters. This is a first release.

Documentation

Installation

Install packages or binaries

• <a href=https://github.com/eltaline/ctrl/releases>Download</a>

systemctl enable ctrl && systemctl start ctrl

Configuring and using cTRL server

Default credentials: admin:swordfish

Password generation: echo -n "mypassword" | sha512sum

In most cases it is enough to use the default configuration file. A full description of all product parameters is available here: <a href="/OPTIONS.md">Options</a>

This guide uses UUIDs. But the client can set task identifiers in any format.

General methods

Run a task or task list in realtime with pending execution

curl -X POST -H "Auth: login:pass" -H "Content-Type: application/json" -d @task.json http://localhost/run

Queuing a task or task list

curl -X POST -H "Auth: login:pass" -H "Content-Type: application/json" -d @task.json http://localhost/task

Getting a task from the received queue

curl -H "Auth: login:pass" "http://localhost/show?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=received"

Getting all tasks from the received queue

curl -H "Auth: login:pass" "http://localhost/show?queue=received"

Getting a task from the working queue

curl -H "Auth: login:pass" "http://localhost/show?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=working"

Getting all tasks from the working queue

curl -H "Auth: login:pass" "http://localhost/show?queue=working"

Getting a task from the list of completed tasks

curl -H "Auth: login:pass" "http://localhost/show?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=completed"

Getting all tasks from the list of completed tasks

curl -H "Auth: login:pass" "http://localhost/show?queue=completed"

Deleting a task from the received queue

curl -H "Auth: login:pass" "http://localhost/del?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=received"

Deleting all tasks from the received queue

curl -H "Auth: login:pass" "http://localhost/del?queue=received"

Deleting a task from the working queue

curl -H "Auth: login:pass" "http://localhost/del?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=working"

Deleting all tasks from the working queue

curl -H "Auth: login:pass" "http://localhost/del?queue=working"

Deleting a task from the list of completed tasks

curl -H "Auth: login:pass" "http://localhost/del?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=completed"

Deleting all tasks from the list of completed tasks

curl -H "Auth: login:pass" "http://localhost/del?queue=completed"

Input format

Description of fields

• key - an arbitrary unique identifier (colon is not allowed ":")
• type - arbitrary type of task (colon is not allowed ":")
• path - path to change the directory before running the command
• lock - arbitrary lock label
• command - command
• threads - the number of threads for a particular type of task
• timeout - timeout
• ttltime - task lifetime in completed queue
• interval - the interval between starting tasks
• repeaterr - enumeration of errors that require the repeated execution of a task
• repeatcnt - the number of task reruns in case of any error from parameter repeaterr
• interr - enumeration of errors that require kill running task
• intcnt - the number of task reruns in case of any error from parameter interr
• lookout - enable or disable reaction on errors from stdout
• replace - overwrite the same task with the same key and the same type in the received queue

Fields threads/ttltime/interval/repeaterr/repeatcnt/interr/intcnt/lookout/replace are relevant only for tasks queued

Examples of setting tasks

Example for one task through /run:

[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","timeout":15}
]

Example for a task list through /run:

[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","timeout":15},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","timeout":15},
{"key":"3fdf744d-36f1-499d-bd39-90a004ee39f6","type":"mytype","path":"/","lock":"mylock3","command":"echo \"world\" && logger \"world\" && sleep 15","timeout":15}
]

Example for one task through /task:

[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false}
]

Example for a task list through /task:

[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false},
{"key":"3fdf744d-36f1-499d-bd39-90a004ee39f6","type":"mytype","path":"/","lock":"mylock3","command":"echo \"world\" && logger \"world\" && sleep 15","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false}
]

Output format

Description of fields

• key - an arbitrary unique identifier
• time - task receive time
• type - arbitrary type of task
• path - path to change the directory before running the command
• lock - arbitrary lock label
• command - command
• threads - the number of threads for a particular type of task
• timeout - timeout
• ttltime - task lifetime in completed queue
• interval - the interval between starting tasks
• repeaterr - enumeration of errors that require the repeated execution of a task
• repeatcnt - the number of task reruns in case of any error from parameter repeaterr
• interr - enumeration of errors that require kill running task
• intcnt - the number of task reruns in case of any error from parameter interr
• lookout - enable or disable reaction on errors from stdout
• replace - overwrite the same task with the same key and the same type in the received queue
• stdcode - not used at the moment
• stdout - standard output
• errcode - error code
• stderr - standard error output
• runtime - runtime in milliseconds
• delcode - error code when deleting a task
• delerr - error message when deleting a task

Output example

Output of completed tasks:

[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","time":1589737139,"type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"hello\n","errcode":0,"stderr":"","runtime":10010.669069},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","time":1589737139,"type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"great\n","errcode":124,"stderr":"signal: kille