Layerform
Layerform helps engineers create reusable environment stacks using plain .tf files. Ideal for multiple "staging" environments.
Install / Use
/learn @briefercloud/LayerformREADME
<p align="center"> <a href="https://layerform.dev"> <picture> <source width="320px" media="(prefers-color-scheme: dark)" srcset="./assets/img/layerformspawn-dark-sm.png"> <source width="320px" media="(prefers-color-scheme: light)" srcset="./assets/img/layerformspawn-light-sm.png"> <img width="320px" alt="layerform logo" src="./assets/img/layerformspawn-light-sm.png"> </picture> </a> </p> <h1 align="center"> Layerform </h1> <p align="center"> <strong> Layerform helps engineers create reusable environment stacks using plain `.tf` files. </strong> </p> <br> <h4 align="center"> <a href="https://layerform.dev">Home Page</a> | <a href="https://docs.layerform.dev">Documentation</a> | <a href="https://discord.gg/daGzchUGDt">Discord</a> </h4> <p align="center"> <a href="https://github.com/ergomake/layerform/blob/main/LICENSE"> <img src="https://img.shields.io/github/license/ergomake/layerform" alt="Layerform is released under the GNU GPLv3 license." /> </a> <a href="https://discord.gg/daGzchUGDt"> <img src="https://img.shields.io/discord/1055836143733706874" alt="Discord Chat" /> </a> <a href="https://twitter.com/intent/follow?screen_name=GetErgomake"> <img src="https://img.shields.io/twitter/follow/GetErgomake.svg?label=Follow%20@GetErgomake" alt="Follow @GetErgomake" /> </a> </p> <br>
After HashiCorp's announcement that they're not using the MPL license anymore, we'll be building on top of OpenTF.
<br>What is Layerform?
Layerform is a Terraform wrapper that helps engineers build reusable infrastructure using plain Terraform files.
To enable reuse, Layerform introduces the concept of layers. Each layer contains some infrastructure and can be stacked up on top of another layer.
In addition to being much easier to use, Layerform allows teams to reuse core-pieces of infrastructure. That way, development infrastructure is much cheaper and quicker to spin up. With Layerform, Engineers only spawn the infrastructure layers they need.
<p align="center"> <picture> <source width="600px" media="(prefers-color-scheme: dark)" srcset="./assets/img/dev-environments-dark.png"> <source width="600px" media="(prefers-color-scheme: light)" srcset="./assets/img/dev-environments.png"> <img width="600px" alt="layerform logo" src="./assets/img/dev-environments.png"> </picture> </p><br>For those wanting development environments: we don't want to run your text-editor. Layerform is the standard tool for development infrastructure. You can keep using your text-editors, IDEs, and other local development directly on your machine.
Why use Layerform
<picture align="right"> <source align="right" width="384px" media="(prefers-color-scheme: dark)" srcset="./assets/img/crocodile-spawn-dark.png"> <source align="right" width="384px" media="(prefers-color-scheme: light)" srcset="./assets/img/crocodile-spawn.png"> <img align="right" width="384px" alt="layerform logo" src="./assets/img/crocodile-spawn.png"> </picture>Cheaper and quicker development infrastructure
Layerform is much cheaper and quicker than spinning up an entire staging environment for each engineer.
When using Layerform, engineers can share the same core-pieces of infrastructure, and only spin up the layers they need on top of it.
For example, if you run applications in a Kubernetes cluster, you don't have to create a brand new cluster for each engineer's environment. Instead, you can reuse the same cluster and have multiple engineers spin up their applications on top of it.
<br>It's just like production, every time
Layerform's environments are just like production because they are spun up from plain .tf files.
Whatever you can set up using a .tf file, you can set up in a Layerform layer.
That way, your development infrastructure will be just like production, including Lambdas, DynamoDB instances, and whatever else that you need.
<br>You own the infrastructure and permissions
Layerform runs in your infrastructure. Layerform will store state and spin up resources in your cloud providers. Therefore, you have full control over who gets access to what, and what resources are currently running, and how much they cost.
<br>Encapsulation / Isolation of concerns
By breaking infrastructure into layers, your organization can define clearer boundaries between teams. Consequently, it will be easier to mirror your organization's structure into your system's structure.
<p align="center"> <picture> <source width="600px" media="(prefers-color-scheme: dark)" srcset="./assets/img/layers-vs-org-dark.png"> <source width="600px" media="(prefers-color-scheme: light)" srcset="./assets/img/layers-vs-org.png"> <img width="600px" alt="layerform logo" src="./assets/img/layers-vs-org.png"> </picture> </p> <br>Cost attribution and cost savings
In addition to saving costs by reusing infrastructure, Layerform allows you to automatically track costs for each layer instance.
When applying layers, Layerform will automatically tag the resources it creates with the actual name assigned to the layer instance. If you have production and development base layers, for example, each of those two will contain the tags layerform_layer_name and layerform_layer_instance with their respective names.
That way, Layerform can recursively traverse layers' resources to collect cost management information. Consequently, it will be able to tell the cost of your whole production and development layers, as well as an aggregate cost report of everything on top of those layers.
Getting started
First, install the Layerform CLI.
$ go install github.com/ergomake/layerform@latest
Then, create the Terraform files you'll use to create each layer of infrastructure. In the example below, we have two layers: eks, which is the "base" layer, and services.
layerform/
├─ services/
│ ├─ pods.tf
│ ├─ outputs.tf
│ └─ inputs.tf
├─ eks/
│ ├─ eks.tf
│ ├─ vpc.tf
│ ├─ inputs.tf
│ └─ outputs.tf
├─ services.tf
└─ eks.tf
Once you have your infrastructure defined as code, you'll create the layer definitions that the CLI will use when spawning instances of each layer.
{
"layers": [
{
"name": "base",
"files": ["./layerform/eks.tf", "./layerform/eks/**"]
},
{
"name": "services",
"files": ["./layerform/services.tf", "./layerform/services/**"],
"dependencies": ["base"]
}
]
}
Now, configure the place in which the generated layer definitions will be saved.
# In config inside ~/.layerform
currentContext: remote-context
contexts:
remote-context:
type: s3
bucket: layerform-bucket-example
Finally, you should provision S3 with your layer definitions using layerform configure.
The Layerform CLI will then take care of creating unique IDs for each layer and sending the Terraform files' contents to the Layerform back-end, which, in this case, is an S3 bucket.
After provisioning layer definitions, you can use layerform spawn <definition_name> <desired_id> to create an instance of that particular layer.
$ layerform spawn services my-dev-infra
Each instance of a layer contains all the pieces of infrastructure defined within that layer's files.
<p align="center"> <picture> <source width="500px" media="(prefers-color-scheme: dark)" srcset="./assets/img/default-base-layer-dark.png"> <source width="500px" media="(prefers-color-scheme: light)" srcset="./assets/img/default-base-layer.png"> <img width="500px" alt="layerform logo" src="./assets/img/default-base-layer.png"> </picture> </p>In this example, running that command will cause Layerform to also create an instance of the underlying eks layer and assign it the ID default.
To spawn yet another services layer, just run layerform spawn services another-dev-infra. By default, Layerform will try to use underlying layers whose ID is default as base layers.
As a general rule, underlying layers are always the ones whose ID is default. To specify the desired ID for each underlying layer, you'll have to use the --base parameter. For example:
# Creates:
# 1. An eks layer with ID "one"
# 2. A services layer with ID "two"
$ layerform spawn services two --base "eks=one"
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="./assets/img/one-two-layers-dark.png">
<source media="(prefers-color-scheme: light)" srcset="./assets/img/one-two-layers.png">
<img alt="layerform logo" src="./assets/img/one-two-layers.png">
</picture>
</p>
<br>
Layer immutability and layer rebasing
A layer can only mutate itself or the layers above. For example, if you have a base layer and a backend layer, the backend layer's Terraform files will not be able to mutate any infrastructure in a base layer instance. Still, the base layer files can mutate any instances of the layers above it.
The way Layerform prevents undesirable mutations is by analyzing each terraform plan and detecting whether any mutation's target belongs to an underlying layer.
The reason Layerform prevents a layer from mutating its underlying layer is to avoid breaking sibling pieces of infrastructure.
This design allows for plat
