<p align="center"> <a alt="lines" href="https://rosineygp.github.io/lines"> <img src="docs/assets/logo.png"/> </a> </p>

Lines

A pure bash clojureish CI pipeline.

Main Features

• Complete CI engine
• Execute local or remote
• Pure data syntax edn
• Clojure script syntax
• Easy command line integration like (docker, kubectl, apt)
• Concurrency execution
• Modular and extensible

Why?

• Alternative method to install clojure, using "clojure", but without clojure
• Tired to write yaml every day
• Create something cool with Fleck

Table of contents

• Installation
• Job keywords
• Modules
  • shell
  • docker
  • scp
  • template
  • user module
• EDN Pipeline
  • Targets file
• Clojure Pipeline
  • Lines functions
• Environment vars
• Extensions
• Development

Installation

requirements

• bash 4
• coreutils
• git
• docker ¹
• ssh ²
• scp ²

¹ docker module<br>² remote execution

install script

# download

curl -L https://github.com/rosineygp/lines-sh/releases/latest/download/lines > lines
chmod +x lines
sudo mv lines /usr/bin/lines

minimal usage

Create a file .lines.edn with followin data.

[{:apply ["echo hello world!"]}]

Just execute command lines

name: lines *******************************************************************
target: local
start: ter 22 dez 2020 21:38:01 -03
  cmd: echo hello world!
    hello world!
  exit-code: 0
status: true
finished: ter 22 dez 2020 21:38:04 -03 ****************************************

Job keywords

A job is defined as a hashmap of keywords. The commons keywords available for job are:

| keyword | type | description | |-------------------------------|---------|------------------------------------------------------------| | apply | array | the tasks that job will handler | | name | string | optional job name | | module | string | module name | | target | hashmap | where the job will run | | vars | hashmap | set env vars (branch name changes conforme current branch) | | args | hashmap | arguments modules, each module has own args | | ignore-error | boolean | when job fail the execution continue | | retries | integer | number of retries if job fail, max 2 |

apply

Is the only required keyword that job needs. It's an array of objects that will be executed by a module. Each element of array is a tasks and the job can handler N tasks. If any tasks has a exit code different than 0, the job will stop and throw the error. ignore-error and retries helps to handler errors.

{:apply ["uname -a"
         "make build"]}

using default module shell, apply is a array of strings.

{:module "scp"
 :apply [{:src "program.tar.gz" :dest "/tmp/program.tar.gz"}
         {:src "script.sh"      :dest "/tmp/script.sh"}]}

scp uses a list of hashmaps

name

name is just a label for the job.

{:name "install curl"
 :module "shell"
 :apply ["apk add curl"]}

module

module is the method executed by job (default is shell)

{:module "shell"
 :apply ["whoami"]}

Docker module

{:module "docker"
 :args {:image "node"}
 :apply ["npm test"]}

the builtin modules are:

| module | description | |----------|--------------------------------------------------------------| | shell | execute shell commands | | docker | start a docker instance and execute shell commands inside it | | template | render lines template and copy to it to destiny | | scp | copy files over scp |

is possible create custom modules

target

Host target is the location where the job will run. If any target passed the job will run at localhost.

{:target {:label "web-server"
          :host "web.local.net"
          :port 22
          :method "ssh"}}

Targets can be defined in separated file, during execution is possible to merge data with job and execute the same job in n hosts.

| keywords | description | |----------|-----------------------------------| | label | host label, just for identify | | host | ip or fqdn for access host | | user | login user | | port | method port 22 is default | | method | connection method, ssh is default |

Is possible set another keywords for filter like group, dc or any other value you need to organizer targets.

After job executed it return themself with result values

vars

Variables will be inject in environment during tasks execution.

{:vars {MY_VAR_0 "lines"
        MY_VAR_1 "go"}
 :apply ["echo $MY_VAR_0"
         "echo $MY_VAR_1"]}

BRANCH_NAME and BRANCH_NAME_SLUG are inject in environment.

args

Args is the parameters of modules.

{:args {:sudo true}
 :apply ["apt-get update"
         "apt-get install htop -y]}

All tasks will run with sudo

ignore-error

If some task fail, lines will not stop the pipeline, just return the current task failed.

{:ignore-error true
 :apply ["whoami"
         "exit 1"
         "dpkg -l"]}

The tasks after error will not be executed.

retries

If some task fail, retry will run it again.

{:retries 2
 :apply ["ping -c 1 my-host"]}

The max retries are 2, but it can be increase setting LINES_JOB_MAX_ATTEMPTS at environment vars.

Modules

shell

Is it the default module, just spawn scripts to shell.

{:module "shell"
 :apply ["date"]}

Command will be executed inside a heredoc block

<command_header> bash -s <<-'LINES-BLOCK-EOF'
export <env vars>;
commands
LINES-BLOCK-EOF

arguments

| keyword | type | description | |------------|---------|-----------------------------------------------------------| | sudo¹ | boolean | apply commands using sudo | | user¹ | string | change current user | | entrypoint | array | change initial entry command (default is ["bash" "-s"]) |

¹ needs pre configured sudoers (without password)

docker

Create a docker instance and execute commands inside it.

single instance

{:module "docker"
 :apply ["whoami"]}

• start docker instance with default image (alpine)
• run command whoami inside container

services

{:module "docker"
 :args {:image "ubuntu"
        :services [{:image "nginx"
                    :alias "nginx"}]}
 :apply ["apt-get update"
         "apt-get install curl -y"
         "curl http://nginx"]}

• start docker instance with nginx image as a service and set network alias as nginx
• start another docker instance with ubuntu image
• install ubuntu packages
• execute curl at service from ubuntu instance

download artifacts

Download files or directory from a docker instance.

{:module "docker"
 :args {:artifacts {:paths ["file"
                            "directory"]}}
 :apply ["touch file"
         "mkdir directory"
         "touch directory/file"]}

arguments

| keyword | type | description | |-------------------------------------|---------|--------------------------------------------------------| | image | string | docker instance path name | | entrypoint | array | change initial entry command (default is sh) | | privileged | boolean | run job with privileged access and mount docker socket | | services | array | services description | | artifacts | hasmap | download artifact from docker instance |

services description

Services is an array of hashmaps, is possible up N services with docker module, the following keywords can be used to build a service.

| keyword | type | description | |------------|---------|---------------------------------------------------| | image | string | docker instance path name | | vars | hashmap | like job vars but exclusive from instance service | | alias | string | network alias name, otherwise slug image name | | entrypoint | string | service entrypoint string, otherwise '' |

artifacts description

Download a files or directories from a docker instance.

| keyword | type | description | |---------|-------|--------------------------------------| | paths | array | file or folder relative or full path |

scp

Copy files and folders to remote host over scp.

{:module "scp"
 :apply [{:src "./dist/command.bin"
          :dest "/usb/bin/command"}]}

apply arguments

| keyword | type | description | |-----------|---------|---------------------------------| | src | string | file or directory source