GoQuarkChain

Go implementation of QuarkChain.

QuarkChain is a sharded blockchain protocol that employs a two-layer architecture - one extensible sharding layer consisting of multiple shard chains processing transactions and one root chain layer securing the network and coordinating cross-shard transactions among shard chains.

Features

• Cluster implementation allowing multiple processes / physical machines to work together as a single full node
• State sharding dividing global state onto independent processing and storage units allowing the network capacity to scale linearly by adding more shards
• Cross-shard transaction allowing native token transfers among shard chains
• Adding shards dynamically to the network
• Support of different mining algorithms on different shards
• P2P network allowing clusters to join and leave anytime with encrypted transport
• Fully compatible with Ethereum smart contract

Design

Check out the Wiki to understand the design of QuarkChain.

Development Setup

The following instructions are based on clean Ubuntu 18.04. For other operating systems, please refer to this FAQ. If you prefer to use Docker, you can run a cluster inside Docker container.

Setup Go Environment

Goquarkchain requires golang sdk >= 1.13. You can skip this step if your environment meets the condition.

# take go1.14.12 for example:
wget https://studygolang.com/dl/golang/go1.14.12.linux-amd64.tar.gz
sudo tar xvzf go1.14.12.linux-amd64.tar.gz -C /usr/local

Append the following environment variables to ~/.profile. NOTE goproxy and go.mod are used.

export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
export GOPROXY=https://goproxy.io
export GO111MODULE=on

Apply the changes immediately

source ~/.profile

Check Go installation

go version #go version go1.14.12 linux/amd64

Setup RocksDB Environment

Before install RocksDB, you'll need to install the following dependency packages.

sudo apt-get update
sudo apt-get install -y git build-essential make g++ swig libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev

Install RocksDB. Version v6.1.2 is recommended.

git clone -b v6.1.2 https://github.com/facebook/rocksdb.git
cd rocksdb
sudo make shared_lib
sudo make install-shared

Append the following environment variables to ~/.profile

export CGO_CFLAGS=-I/usr/local/include
export CGO_LDFLAGS="-L/usr/local/lib -lrocksdb -lstdc++ -lm -lz -lbz2 -lsnappy"
export LD_LIBRARY_PATH=/usr/local/lib

Apply the changes immediately

source ~/.profile

Setup GoQuarkChain

mkdir -p $GOPATH/src/github.com/QuarkChain && cd $_
git clone https://github.com/QuarkChain/goquarkchain.git
#build qkchash
cd goquarkchain/consensus/qkchash/native
sudo g++ -shared -o libqkchash.so -fPIC qkchash.cpp -O3 -std=gnu++17 && make

Run all the unit tests under goquarkchain to verify the environment is correctly set up:

cd $GOPATH/src/github.com/QuarkChain/goquarkchain
go test ./...

Running Clusters

The following instructions will lead you to run clusters step by step.

If you need to join mainnet and mine quickly, we recommend following this instruction to start a cluster using docker.

If you need to deploy clusters onto many hosts, a better option would be Use Deploy Tool to Start Clusters, which is based on pre-built Docker image.

Build Cluster

Build GoQuarkChain cluster executable:

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
go build

Running a single cluster for local testing

To run a local cluster which does not connect to anyone else, start each slave in different terminals with its ID specified in SLAVE_LIST of the json config.

The following example has 2 slaves with 1 shard for each:

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
./cluster --cluster_config ../../tests/testnet/egconfig/cluster_config_template.json --service S0

In anther terminal,

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
./cluster --cluster_config ../../tests/testnet/egconfig/cluster_config_template.json --service S1

Start master in a third terminal

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
./cluster --cluster_config ../../tests/testnet/egconfig/cluster_config_template.json

Or, instead of running above 3 commands, use the following command to run the processes in background:

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
./run_cluster.sh  ../../tests/testnet/egconfig/cluster_config_template.json

Run a Cluster Inside Docker

Using pre-built Docker image(quarkchaindocker/goquarkchain), you can run a cluster inside Docker container without setting up environment step by step.

Refer to Docker docs if Docker is not yet installed on your machine.

Run the following commands to pull and start a container:

# specify a version tag if needed; use 'latest' for latest code 
sudo docker pull quarkchaindocker/goquarkchain:<version tag>
sudo docker run -it quarkchaindocker/goquarkchain:<version tag>

Now you are inside Docker container and are ready to start cluster services with a sample cluster config:

root@<container ID>:/go/src/github.com/QuarkChain/goquarkchain/cmd/cluster#./run_cluster.sh  ../../tests/testnet/egconfig/cluster_config_template.json

Check logs to see if the cluster is running successfully.

NOTE if you need to start the service with websocket, you can need to start the cluster using ./run_cluster_with_websocket.sh instead of using ./run_cluster.sh.

Next you can try to start a simulate mining.

NOTE if you need the services available outside of the Docker host, you can publish the related ports using -p flag when start Docker:

sudo docker run -it -p 38291:38291 -p 38391:38391 -p 38491:38491 -p 38291:38291/udp quarkchaindocker/goquarkchain

NOTE if websocket service is needed, add -p 38590-38597:38590-38597 to docker ports mapping.

And config rpc listening to 0.0.0.0 when start master service:

./cluster --cluster_config $CLUSTER_CONFIG_FILE --json_rpc_host 0.0.0.0 --json_rpc_private_host 0.0.0.0

Start Metamask API Service

If you want to use metamask to connect your node, you need to start service under folder /cmd/eth_api using following command.

<goquarkchain>/cmd/eth_api# go run main.go --config ./mainnet.json

Join QuarkChain Network

If you are joining a testnet or mainnet, make sure ports are open and accessible from outside world: this means if you are running on AWS, open the ports (default both UDP and TCP 38291) in security group; if you are running from a LAN (connecting to the internet through a router), you need to setup port forwarding for UDP/TCP 38291.

Before running, you'll need to set up the configuration of your network, which all nodes need to be aware of and agree upon. We provide an example config JSON which you can use as value of --cluster_config flag to start cluster service and connect to mainnet:

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
./cluster --cluster_config ../../mainnet/singularity/cluster_config_template.json (--service $SLAVE_ID)

NOTE that many parameters in the config are part of the consensus, please be very cautious when changing them. For example, COINBASE_AMOUNT is one such parameter, changing it to another value effectively creates a fork in the network.

NOTE if you run into the issue "Too many open files", which may occur while your cluster sync to mainnet, try to increase the limitation for current terminal by executing the following command before start cluster:

ulimit -HSn 102400

Running multiple clusters with P2P network on different machines

To run a private network, first start a bootstrap cluster, then start other clusters with bootnode URL to connect to it.

Start Bootstrap Cluster

First start each of the slave services as in Running a single cluster for local testing:

cd $GOPATH/src/github.com/QuarkChain/goquarkchain/cmd/cluster
./cluster --cluster_config ../../tests/testnet/egconfig/cluster_config_template.json --service $SLAVE_ID

Next, start the master service of bootstrap cluster, optionally providing a private key:

./cluster --cluster_config $CLUSTER_CONFIG_FILE --privkey $BOOTSTRAP_PRIV_KEY

You can copy the full boot node URL from the console output in format: enode://$BOOTSTRAP_PUB_KEY@$BOOTSTRAP_IP:$BOOTSTRAP_DISCOVERY_PORT.

Here is an example:

INFO [11-04|18:05:54.832] Started P2P networking self=enode://011bd77918a523c2d983de2508270420faf6263403a7a7f6daf1212a810537e4d27787e8885d8c696c3445158a75cfe521cfccab9bc25ba5ac6f8aebf60106f1@127.0.0.1:38291

NOTE if your clusters are cross-internet, you need to replace $BOOTSTRAP_IP part with PUBLIC ip address when used as --bootnodes flag for other clusters.

NOTE if private key is not provided, the boot node URL will change at each restart of the service.

NOTE the PRIV_KEY field of P2P section in cluster config file has same effect and can be overridden by `--privke