OpenFn/Lightning

Lightning ⚡ (aka "OpenFn v2") is a workflow automation platform that's used to automate critical business processes and integrate information systems. From last-mile services to national-level reporting, it boosts efficiency & effectiveness while enabling secure, stable, scalable interoperability and data integration at all levels.

Use it online at app.openfn.org

Explore in a sandbox on demo.openfn.org

Or learn more at docs.openfn.org

OpenFn Lightning is:

the latest version of OpenFn: first launched in 2014, it's been tried and tested by NGOs and governments in 40+ countries

fully open source: there's no "community edition" and "premium edition", you get the same product whether you are self-hosting or using the OpenFn.org software-as-a-service

the leading DPGA certified Digital Public Good for workflow automation

a Digital Square certified Global Good for Health

Demo
Features
Getting Started
Security and Standards
Contribute to this project
Server Specs for Self-Hosting
Benchmarking
Troubleshooting
Support

Demo

Watch a short demo video or explore a public sandbox with the login details below, but please note that this deployment is reset every night at 12:00:00 UTC and is 100% publicly accessible. Don't build anything you want to keep, or keep private!

username: demo@openfn.org
password: welcome12345

Features

Build

Plan and build workflows using Lightning's visual interface to quickly define when, where and what you want your automation to do.

Use our CLI to quickly build, edit and deploy projects from the comfort of your own code editor.

Monitor

Monitor all workflow activity in one place.

Filter and search runs to identify issues that need addressing and follow how a specific request has been processed
Configure alerts to be notified on run failures
Receive a project digest for a daily/weekly/monthly summary of your project activity

Manage

Manage users and access by project.

Roles and permissions

Authorization is a central part of Lightning. As such, users are given different roles which determine what level of access they have for resources in the application. For more details about roles and permissions in Lightning, please refer to our documentation.

Roadmap

View our public GitHub project to see what we're working on now and what's coming next.

Getting Started

If you only want to RUN Lightning on your own server, we recommend using Docker.
If you want to DEPLOY Lightning, we recommend Docker builds and Kubernetes.
If you want to CONTRIBUTE to the project, we recommend running Lightning on your local machine.

Run via Docker

Install the latest version of Docker
Clone this repo using git
Setup PostgreSQL database with: docker compose build && docker compose run --rm web mix ecto.migrate
Run Lightning and PostgresSQL with: docker compose up

By default the application will be running at localhost:4000.

See "Problems with Docker" for additional troubleshooting help. Note that you can also create your own docker-compose.yml file, configuring a postgres database and using a pre-built image from Dockerhub.

Deploy on external infrastructure

Head to the Deploy section of our docs site to get started.

For technical guidelines, see deployment considerations for more detailed information.

Dev on Lightning locally

Clone the repo and optionally set ENVs

git clone git@github.com:OpenFn/Lightning.git # or from YOUR fork!
cd Lightning
cp .env.example .env # and adjust as necessary!

Take note of database names and ports in particular—they've got to match across your Postgres setup and your ENVs. You can run lightning without any ENVs assuming a vanilla postgres setup (see below), but you may want to make adjustments.

Database Setup

If you're already using Postgres locally, create a new database called lightning_dev, for example.

If you'd rather use Docker to set up a Postgres DB, create a new volume and image:

docker volume create lightning-postgres-data

docker create \
  --name lightning-postgres \
  --mount source=lightning-postgres-data,target=/var/lib/postgresql/data \
  --publish 5432:5432 \
  -e POSTGRES_PASSWORD=postgres \
  postgres:15.3-alpine

docker start lightning-postgres

Elixir & Ecto Setup

We use asdf to configure our local environments. Included in the repo is a .tool-versions file that is read by asdf in order to dynamically make the specified versions of Elixir and Erlang available.

You'll need to install asdf plugins for Erlang, NodeJs, Elixir, and k6. Important: Before installing these plugins, make sure to follow the prerequisite installation steps for each plugin (linked below), as they often require system dependencies to be installed first:

Erlang - Follow the "Before asdf install" section
NodeJs
Elixir
k6

Then follow the installation instructions for each plugin from their respective documentation pages.

We use libsodium for encoding values as required by the Github API. You'll need to install libsodium in order for the application to compile.

For Mac Users:

brew install libsodium cmake

For Debian Users:

sudo apt-get install libsodium-dev cmake

You can find more on how to install libsodium here

asdf install  # Install language versions
./bin/bootstrap

Run the app

Lightning is a web app. To run it in interactive Elixir mode, start the development server by running with your environment variables by running:

iex -S mix phx.server

or if you have set up custom environment variables, run:

env $(cat .env | grep -v "#" | xargs ) iex -S mix phx.server

Once the server has started, head to localhost:4000

Lightning

Install / Use

README