Simverse

This is a tool for lightning network developers. It can generate local simnet clusters of requested size and shape. It uses docker-compose to manage the cluster and provides a set of helper scripts for your convenience.

Features

• supports btcd and bitcoind as backend nodes
• supports lnd, lightningd and eclair as lightning nodes

Workflow

You use a simple DSL script (bash) as a recipe to describe your cluster. Given a recipe, Simverse will generate a bunch of dockerfiles, docker-compose config and other supporting scripts for you. Before generating the files you can tweak your configuration using env variables.

Prerequisites:

On host machine you should have recent versions of:

• bash 4+, git 2+, jq 1.6+
• docker 18+, docker-compose 1.24+

Tested on macOS and Ubuntu 18.10.

Quick start

In a terminal session:

> git clone --recursive https://github.com/darwin/simverse.git
> cd simverse
> ./sv create                                 # first time, it will ask to clone repos, answer yes
...
> ./sv enter
...
_workspace/default > ./dc build
...
_workspace/default > ./dc up                  # this gives you a nice log for whole cluster
...

In a second terminal session:

> cd simverse
> ./sv enter
_workspace/default > btcd1 getinfo
...
_workspace/default > alice getinfo
...
_workspace/default > connect alice bob           # connect Alice's lnd node to Bob's
...
_workspace/default > fund alice 10               # fund Alice with 10 BTC
...
_workspace/default > open_channel alice bob 0.1  # open a LN channel
...
_workspace/default > pay alice bob 0.01          # pay via LN

Here is the default recipe which was used for simnet generation (see recipes/default.sh):

. cookbook/cookbook.sh

prelude

add btcd

add lnd alice
add lnd bob

If you want different configuration, see recipes/readme.md to understand the naming convention of recipes. Then you can create the simverse by adding a parameter to the create script:

./sv create a1k2

This will create a bitcoind and two lightningd configuration simverse.

If you don't want to risk putting your machine on fire, here is a recording:

• note that we used pre-generated ./tmux script to launch everything in one terminal window

Generated simnet folder

A simnet folder contains generated files for your simnet based on a selected recipe.

Typical workflow is to create simnet and then enter it via ./sv enter [simnet_name], then you can

• use ./dc command to manage simnet cluster via docker compose (see generated docker-compose.yml).
• use tools from toolbox and (generated) aliases folders (they should be on $PATH).

An example simnet session

See 'quick start' section above.

Go to simverse root directory. Please open two terminal windows:

• The first one will be for logs
• The second one will be for interaction with running cluster

In the first terminal session

1. Enter simnet via ./sv enter default
2. Build simnet via ./dc build
3. Launch simnet via ./dc up

When you hit CTRL+C, docker will shutdown your simnet and you can rinse, repeat. See tips below in FAQ to learn more.

In the second terminal session

1. (assuming running simnet)
2. Enter simnet via ./sv enter default
3. you can control individual services via their names e.g. alice getinfo or btcd1 getnetworkinfo (see aliases dir). Also note that first lnd node has alias lncli and first btcd node has alias btcctl.
4. you can also use convenience commands from toolbox, for example:

# assuming you have at least one btcd node and lnd nodes alice, bob and charlie (see the recipe 'b1l3.sh')
#
# let's follow the tutorial https://dev.lightning.community/tutorial/01-lncli

fund alice 10
fund bob 10
fund charlie 10

connect alice charlie
connect bob charlie

open_channel alice charlie 0.1
open_channel charlie bob 0.05

# wait for channel discovery to kick in (circa 20s)

pay alice bob 0.01

Alternatively you can use our pre-generated ./tmux script.

Gotchas

• If in troubles, please make sure you have the latest versions tools above, especially docker and docker-compose.
• Simverse tries to enforce host's UID and GID as permissions for simnet user running the services like lnd or btcd. This is quite important for comfortable access to simnet state folders from host machine under some systems. This can fail in case when your current UID(GID) on host is already taken inside Alpine Linux container. In such cases the effective user in the container is disconnected from host user and permissions on state folder might be different. You might need root permission to remove them on host.

macOS notes

• (recommended)keep workspace on APFS-formatted filesystem to benefit from clonefile

FAQ

Is this secure?

No. This is a developer setup and should be never used on a machine connected to open internet. It is expected to be used developer machine on a local network behind NAT/firewall and should never contain sensitive info. Anyways whatever you do, you use it at your own risk.

How do I customize build/runtime parameters for individual docker containers?

By customizing your recipe before adding a node or globally via env variables. See _defaults.sh. For example you can set different DEFAULT_LND_REPO_PATH before each add lnd ... command. This would tweak effective config for following lnd nodes. Read the cookbook.sh bash script for better understanding.

How do I rebuild everything from scratch?

./dc --no-cache build

How do I determine ports mapping to my host machine?

./dc ps (assuming a running simnet)

I'd like to know IP addresses of individual machines in the simnet cluster. How?

list_docker_ips please note that you should not use hard-coded IPs, instead use service names docker assigned to individual machines

Is it possible to launch multiple simnets in parallel?

It is possible to create multiple simnets. But under normal circumstances you are expected to run only one simnet at a time. By default, all simnets use the same port mappings to the host machine, so you would not be able to launch them in parallel. But you can write a simple wrapper scripts which could modify all *_PORT_ON_HOST in _defaults.sh. You can allocate them so that they don't overlap for simnets you need to run in parallel.

I noticed the nodes run in regtest mode, not simnet. Why?

Simnet mode is supported only in btcd and lnd. Bitcoind and c-lightning have regtest which happens to be available in btcd/lnd as well. To allow hybrid simverse clusters we had to use regtest mode everywhere (which works like simnet for our purposes). Only we had to patch btcd to fix some minor issue, because not many people use it this way I guess.

How can I inspect a running container in shell?

./dc exec <container> bash or ./dc exec <container> fish or ./dc exec <container> sh

or ./dc exec <container> <one-off-cmd>

Pro Tip: you might want to add --user root before container name to run the command with unrestricted permissions

Is it possible to keep simverse workspace on my own path?

Yes. Look for SIMVERSE_WORKSPACE env var.

Is it possible to keep projects' repos on my own path?

Yes. Look for SIMVERSE_REPOS env var.

What is a master bitcoin node?

In a cluster of bitcoin nodes we need one which will be special. It will do some privileged tasks like mining or holding mined coins in its associated wallet. By convention, master bitcoin node is always first bitcoin node created. You can get its hostname by running lookup_host 1 role bitcoin. Simverse supports multiple bitcoin node implementations and master node might be either btcd or bitcoind "flavor". Raw cli interface might be slightly different, so we try to hide this in our commands like onchain_balance, chain_height or earn. If you looked at their implementation you would spot code branches for different bitcoin node flavors.

Where is a faucet?

This is not a testnet with faucet so we have to mine coins ourselves. Look for FAUCET_ADDR env var which contains pre-generated bitcoin address which will receive all mined coins. During bitcoin node initialization we import this address into a wallet (see setup-wallet.sh). This way btcctl --wallet sendfrom imported ... can be used to send funds to others in need. We use similar approach when your master bitcoin node is bitcoind.

Please look at toolbox/fund script which can be used for convenient funding of lnd wallets. For example run fund alice 10 to send 10 BTC to Alice's lnd node wallet. The script might decide to call toolbox/earn to mine enough coins and wait for their maturity.

What is the difference between heterogeneous and hybrid simnets?

A homogeneous simnet contains only btcd + lnd nodes or only bitcoind + c-lightning nodes. Genera