Brenda -- Blender render farm software for Amazon Web Services.

Brenda uses Amazon EC2, S3, and SQS to implement a distributed render farm using low-cost EC2 spot instances. Using Brenda, you can accelerate complex render tasks by distributing the work to tens, hundreds, or even thousands of virtual machines in the cloud.

Brenda is specifically designed to take advantage of the lower-cost EC2 spot market and is fault-tolerant against instances being created and terminated at random during the course of a render job, as is often the case with spot market volatility.

You can view James Yonan's Brenda talk at Blender Conference 2013 here:

http://www.youtube.com/watch?v=_Oqo383uviw

Talk notes are in doc/brenda-talk-blendercon-2013.pdf

Brenda includes five tools which are outlined below. To see detailed help for each tool, run the tool with the -h option.

1. brenda-work -- used to create and populate an SQS queue with render tasks. A render task is a short (often one-line) shell script that runs Blender to render a single frame or subframe in a project. Uses the SQS API.

2. brenda-run -- used to start EC2 instances running Brenda, either as on-demand instances or spot instances at a given maximum bid price. Uses the EC2 API.

3. brenda-tool -- used to monitor the operation of an EC2 render farm. It allows an ssh or rsync command to be simultaneously executed on all nodes in the farm. Uses EC2 API and requires a standard unix shell where ssh and rsync are available and can be run from the command line.

4. brenda-ebs -- a simple tool that creates a new EBS volume of a specified size and attaches it to a newly started t1.micro EC2 instance.

5. brenda-node -- worker script to be run on the render farm nodes themselves. It reads tasks from an SQS queue, executes the task (usually render operations), and copies the task products (such as rendered PNG frames) to S3. brenda-node is usually not run directly by the user, but is remotely instantiated by brenda-run. Uses the SQS and S3 APIs.

PLATFORMS SUPPORTED

The Brenda client software is command-line oriented and has currently been tested on Mac OS X and Linux only.

TUTORIAL

This tutorial is intended for use on Mac OS X or Linux.

If you don't have an AWS (Amazon Web Services) account, sign up for one now.

First, install the "boto" python library. This library is used by Brenda to interact with AWS.

Next, download and install Brenda on the client machine.

$ git clone http://github.com/jamesyonan/brenda.git
$ cd brenda
$ python setup.py install

You will need an RSA-based SSH key to access the VMs (virtual machines) that we will spawn using the AWS EC2 service. These VMs are often referred to as "instances", and we will be creating many of these to act as worker nodes in our render farm.

If you have an existing ssh key pair in ~/.ssh/id_rsa.pub and ~/.ssh/id_rsa, the Brenda client software will use it. Otherwise, the software will generate a new key pair on AWS and download the private key to ~/.ssh/id_rsa.brenda

Next, obtain the AWS "Access Key" and "Secret Key" from the AWS management console. These credentials will allow the Brenda client tools to access AWS resources in your account.

We will also need a tool for accessing the AWS S3 file store, because we will be using S3 for two purposes:

1. As a storage location for our Blender project, to allow the render farm nodes to access it.

2. As a storage location for final rendered frames generated by the render farm.

For this, download and install the "s3cmd" tool. You will need to configure s3cmd with your AWS account credentials:

$ s3cmd --configure

At this point, we will copy our Blender project, assets, and other supporting files to the AWS cloud so the render farm can access them. There are two methods to do this, each with their own advantages and disadvantages:

1. Bundle Blender project and supporting files into a zip or tar file and push to S3. S3 is a distributed file storage service hosted by AWS.

   Pros: Relatively easy to set up.

   Cons: Starts to be prohibitively slow as data set gets into the multi-GB range.

2. Create an EBS snapshot containing Blender project and supporting files. An EBS volume is a kind of virtual hard drive that can be attached EC2 instances. An EBS snapshot is a kind of point-in-time copy of an EBS volume that many EC2 instances can concurrently access.

   Pros: Efficient and scalable -- EBS snapshots can be up to 1 TB in size, and Brenda supports attaching up to 66 EBS snapshots to each render farm instance.

   Cons: More involved to set up.

Continuing with the tutorial, we will use the S3 method (1), but note the section "rendering large projects using EBS snapshots" below if your data set is large and you want to use method (2).

Next, we will bundle up our Blender project and save it to S3 so the render farm can access it (make sure that the Blender project uses relative paths for file access so that the render farm instances will be able to follow them).

To do this, create a folder with your .blend file and any other supporting files necessary to render frames, then compress it using tar or zip. For example, supposing that the project directory is called "myproject", run:

$ tar cfzv myproject.tar.gz myproject

Next, create an S3 "bucket" on AWS to store the myproject.tar.gz file we created above. S3 buckets are sort of like folders, but they must have a globally unique name, and they can only contain flat files, not subfolders. You can choose a name here for the project bucket which I will hereafter refer to as PROJECT_BUCKET.

$ s3cmd mb s3://PROJECT_BUCKET

It is possible that the name you chose for PROJECT_BUCKET is already in use by someone else. In this case you will see an error message, and can retry the command using a different name.

We also need another bucket for the render farm to save the rendered frames. We will call this FRAME_BUCKET. Just like PROJECT_BUCKET above, you should select a unique name.

$ s3cmd mb s3://FRAME_BUCKET

Now, we will copy our compressed Blender project file to our S3 project bucket:

$ s3cmd put myproject.tar.gz s3://PROJECT_BUCKET

To verify that the file was copied, list the files in the bucket:

$ s3cmd ls s3://PROJECT_BUCKET

Next, we will create a Brenda configuration file. The Brenda client tools will look for the configuration file in ~/.brenda.conf

Create ~/.brenda.conf now with the following content, making sure to replace PROJECT_BUCKET and FRAME_BUCKET with the names you chose above.

INSTANCE_TYPE=m3.xlarge
BLENDER_PROJECT=s3://PROJECT_BUCKET/myproject.tar.gz
WORK_QUEUE=sqs://FRAME_BUCKET
RENDER_OUTPUT=s3://FRAME_BUCKET
DONE=shutdown

To explain the above configuration settings in detail:

INSTANCE_TYPE describes the type of EC2 instance (i.e. virtual machine) that will make up the render farm. Different instance types offer different levels of performance and cost.

BLENDER_PROJECT is the name of our project file on S3. It can be an s3:// or file:// URL.

WORK_QUEUE is the name of an SQS queue that we will create for the purpose of staging and sequencing the tasks in our render.

RENDER_OUTPUT is the name of an S3 bucket that will contain our rendered frames.

DONE=shutdown tells the render farm instances that they should automatically shut themselves down after the render is complete.

In the next step, we will create the Work Queue for our render farm. A work queue is basically a list of many small scripts that, when run together, will render all of the frames in our project.

Brenda's essential purpose is to accelerate the rendering process by concurrently processing our work queue using tens, hundreds or even thousands of virtual machines.

For example, one of the scripts in a work queue might look like this (to render frame 5 of our project):

blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 5 -e 5 -j 1 -t 0 -a

Suppose our project contains 240 frames. Then the work queue would look like this, where each line is a separate task in the work queue:

blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 1 -e 1 -j 1 -t 0 -a
blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 2 -e 2 -j 1 -t 0 -a
blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 3 -e 3 -j 1 -t 0 -a
 .
 .
 .
blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 238 -e 238 -j 1 -t 0 -a
blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 239 -e 239 -j 1 -t 0 -a
blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s 240 -e 240 -j 1 -t 0 -a

The first step in creating a work queue is to start with a Script Template. A script template describes how to run Blender to accomplish a unit of work. Suppose that we want a unit of work to be the rendering of a single PNG frame. In this case, our script template would be this:

blender -b *.blend -F PNG -o $OUTDIR/frame_###### -s $START -e $END -j $STEP -t 0 -a

Using a text editor, create a file called "frame-template" that contains the above line. This is a simple template that is designed to render one frame at a time (as a more advanced exercise, it is also possible to create a subframe rendering template that will break the smallest unit of render work down to a portion of a frame -- this could be used to accelerate the rendering of a still, or to cut down the time spent processing each unit of work in animations where each frame takes many computer-hours to render).

Brenda includes a tool called "brenda-work" that allows us to easily generate a work queue. Suppose that you want to render the first 240 frames of your project. Use this command to generate the work queue using the "frame-template" file we created above:

$ brenda-work -T frame-temp