Toast 🥂

Toast is a tool for containerizing your workflows such as building and testing a project. You define tasks in a YAML file called a toastfile, and Toast runs them in a container based on a Docker image of your choosing. What constitutes a "task" is up to you: tasks can install system packages, compile an application, run a test suite, or even serve web pages. Tasks can depend on other tasks, so Toast can be understood as a high-level containerized build system.

Here's the toastfile for the example shown above:

image: ubuntu
tasks:
  install_gcc:
    command: |
      apt-get update
      apt-get install --yes gcc

  build:
    dependencies:
      - install_gcc
    input_paths:
      - main.c
    command: gcc main.c

  run:
    dependencies:
      - build
    command: ./a.out

Toast caches each task by committing the container to an image. The image is tagged with a cryptographic hash of the shell command for the task, the contents of the files copied into the container, and all the other task inputs. This hash allows Toast to skip tasks that haven't changed since the last run.

In addition to local caching, Toast can use a Docker registry as a remote cache. You, your teammates, and your continuous integration (CI) system can all share the same remote cache. Used in this way, your CI system can do all the heavy lifting like building and installing dependencies so you and your team can focus on development.

Related tools:

• Docker Compose: Docker Compose is a convenient Docker-based development environment which shares many features with Toast. However, it doesn't support defining tasks (like lint, test, run, etc.) or remote caching.
• Nix: Nix achieves reproducible builds by leveraging ideas from functional programming rather than containerization. We're big fans of Nix. However, Nix requires a larger commitment compared to Toast because you have to use the Nix package manager or write your own Nix derivations. For better or worse, Toast allows you to use familiar idioms like apt-get install ....

To prevent Docker images from accumulating on your machine when using Docker-related tools such as Toast or Docker Compose, we recommend using Docuum to perform least recently used (LRU) image eviction.

Tutorial

Defining a simple task

Let's create a toastfile. Create a file named toast.yml with the following contents:

image: ubuntu
tasks:
  greet:
    command: echo 'Hello, World!' # Toast will run this in a container.

Now run toast. You should see the following:

If you run it again, Toast will find that nothing has changed and skip the task:

Toast caches tasks to save you time. For example, you don't want to reinstall your dependencies every time you run your tests. However, caching may not be appropriate for some tasks, like running a development server. You can disable caching for a specific task and all tasks that depend on it with the cache option:

image: ubuntu
tasks:
  greet:
    cache: false # Don't cache this task.
    command: echo 'Hello, World!'

Adding a dependency

Let's make the greeting more fun with a program called figlet. We'll add a task to install figlet, and we'll change the greet task to depend on it:

image: ubuntu
tasks:
  install_figlet:
    command: |
      apt-get update
      apt-get install --yes figlet

  greet:
    dependencies:
      - install_figlet # Toast will run this task first.
    command: figlet 'Hello, World!'

Run toast to see a marvelous greeting:

Importing files from the host

Here's a more realistic example. Suppose you want to compile and run a simple C program. Create a file called main.c:

#include <stdio.h>

int main(void) {
  printf("Hello, World!\n");
  return 0;
}

Update toast.yml to compile and run the program:

image: ubuntu
tasks:
  install_gcc:
    command: |
      apt-get update
      apt-get install --yes gcc

  build:
    dependencies:
      - install_gcc
    input_paths:
      - main.c # Toast will copy this file into the container before running the command.
    command: gcc main.c

  run:
    dependencies:
      - build
    command: ./a.out

Notice the input_paths array in the build task. Here we're copying a single file into the container, but we could instead import the entire directory containing the toastfile with .. By default, the files will be copied into a directory called /scratch in the container. The commands will be run in that directory as well.

Now if you run toast, you'll see this:

For subsequent runs, Toast will skip the task if nothing has changed. But if you update the greeting in main.c, Toast will detect the change and rerun the build and run tasks on the next invocation.

Exporting files from the container

A common use case for Toast is to build a project. Naturally, you might wonder how to access the build artifacts produced inside the container from the host machine. It's easy to do with output_paths:

image: ubuntu
tasks:
  install_gcc:
    command: |
      apt-get update
      apt-get install --yes gcc

  build:
    dependencies:
      - install_gcc
    input_paths:
      - main.c
    output_paths:
      - a.out # Toast will copy this file onto the host after running the command.
    command: gcc main.c

When Toast runs the build task, it will copy the a.out file to the host.

Passing arguments to a task

Sometimes it's useful for tasks to take arguments. For example, a deploy task might want to know whether you want to deploy to the staging or production cluster. To do this, add an environment section to your task:

image: ubuntu
tasks:
  deploy:
    cache: false
    environment:
      CLUSTER: staging # Deploy to staging by default.
    command: echo "Deploying to $CLUSTER..."

When you run this task, Toast will read the value from the environment:

If the variable doesn't exist in the environment, Toast will use the default value:

If you don't want to have a default, set it to null:

image: ubuntu
tasks:
  deploy:
    cache: false
    environment:
      CLUSTER: null # No default; this variable must be provided at runtime.
    command: echo "Deploying to $CLUSTER..."

Now if you run toast deploy without specifying a CLUSTER, Toast will complain about the missing variable and refuse to run the task.

Environment variables listed in a task are also set for any tasks that run after it.

Running a server and mounting paths into the container

Toast can be used for more than just building a project. Suppose you're developing a website. You can define a Toast task to run your web server! Create a file called index.html with the following contents:

<!DOCTYPE html>
<html>
  <head>
    <title>Welcome to Toast!</title>
  </head>
  <body>
    <p>Hello, World!</p>
  </body>
</html>

We can use a web server like nginx. The official nginx Docker image will do, but you could also use a more general image and define a Toast task to install nginx.

In our toast.yml file, we'll use the ports field to make the website accessible outside the container. We'll also use mount_paths rather than input_paths so we can edit the web page without having to restart the server.

image: nginx
tasks:
  serve:
    cache: false # It doesn't make sense to cache this task.
    mount_paths:
      - index.html # Updates to this file will be visible inside the container.
    ports:
      - 3000:80 # Expose port 80 in the container as port 3000 on the host.
    location: /usr/share/nginx/html/ # Nginx will serve the files in here.
    command: nginx -g 'daemon off;' # Run in foreground mode.

Now you can use Toast to run the server:

Configuring the shell

It's often desirable to configure the shell in some way before running any commands. Shells are typically configured with so-called "startup files" (e.g., ~/.bashrc). However, many shells skip loading such configuration files when running in non-interactive, non-login mode, which is how the shell is invoked by Toast. Toast provides an alternative mechanism to configure the shell that doesn't require creating any special files or invoking the shell in a particular way.

Consider the following toastfile which uses Bash as the shell, since that's the default preferred login shell in Ubuntu:

image: ubuntu
tasks:
  install_figlet:
    command: |
      apt-get update
      apt-get install --yes figlet

What happens if apt-get update fails? Due to the way Bash