Introduction

Btrbk is a backup tool for btrfs subvolumes, taking advantage of btrfs specific capabilities to create atomic snapshots and transfer them incrementally to your backup locations.

The source and target locations are specified in a config file, which allows to easily configure simple scenarios like "laptop with locally attached backup disks", as well as more complex ones, e.g. "server receiving backups from several hosts via ssh, with different retention policies".

Key Features:

Atomic snapshots
Incremental backups
Flexible retention policy
Backups to multiple destinations
Transfer via ssh
Robust recovery from interrupted backups (for removable and mobile devices)
Archive to offline storage
Encrypted backups to non-btrfs storage
Wildcard subvolumes (useful for docker and lxc containers)
Transaction log
Comprehensive list and statistics output
Resolve and trace btrfs parent-child and received-from relationships
List file changes between backups
Calculate accurate disk space usage based on block regions

Btrbk is designed to run as a cron job for triggering periodic snapshots and backups, as well as from the command line (e.g. for instantly creating additional snapshots).

Installation

Btrbk is a single perl script, and does not require any special installation procedures or libraries. Download the latest btrbk source tarball, or try latest master:

wget https://raw.githubusercontent.com/digint/btrbk/master/btrbk
chmod +x btrbk
sudo ./btrbk ls /

For more information, read the installation documentation.

Prerequisites

btrfs-progs: Btrfs filesystem utilities >= v4.12
Perl interpreter: Probably already installed on your system
OpenSSH: If you want to transfer backups from/to remote locations
mbuffer: If you want rate limiting and progress bars

Synopsis

Please consult the btrbk(1) man-page provided with this package for a full description of the command line options.

Configuration

Before running btrbk, you will need to create a configuration file. You might want to take a look at btrbk.conf.example provided with this package. For a detailed description, please consult the btrbk.conf(5) man-page.

After a configuration change, it is highly recommended to check it by running btrbk with the -n,--dry-run option:

# btrbk -c /path/to/myconfig -v -n run

This will read all btrfs information on the source/target filesystems and show what actions would be performed (without writing anything to the disks).

The examples below assume that the btrfs subvolume containing home and rootfs is mounted at /mnt/btr_pool. This is usually the btrfs root subvolume, which always has subvolid=5.

Mounting subvolid=5 is recommended (mandatory for btrbk < v0.32.0) if you want to backup your root filesystem /.

/etc/fstab:

/dev/sda1  /mnt/btr_pool  btrfs  subvolid=5,noatime  0 0

Note that some default btrfs installations (e.g. Ubuntu) use subvolume names @ for rootfs (mounted at /) and @home for /home, as a naming convention. If this is the case on your file system, replace the subvolume declarations in the examples accordingly.

Example: Local Regular Snapshots (time-machine)

The simplest use case is to only create snapshots of your data. This will obviously not protect it against hardware failure, but can be useful for:

protection against inadvertent changes or deletions
keeping past states of copies from rsync or similar tools

Let's assume you need regular snapshots of your home directory, which is located in the subvolume home of the volume /mnt/btr_pool. The snapshots are to be stored in btrbk_snapshots (on the same volume).

/etc/btrbk/btrbk.conf:

timestamp_format        long
snapshot_preserve_min   18h
snapshot_preserve       48h

volume /mnt/btr_pool
  snapshot_dir btrbk_snapshots
  subvolume home

Notice that the target option is not provided, and btrbk will only manage snapshots located on the same volume in snapshot_dir. Btrbk does not create subdirs by default, the snapshot directory must first be created manually:

# mkdir /mnt/btr_pool/btrbk_snapshots

The "volume" section is merely used as a specifier for a base directory, and can be skipped if you prefer to configure everything using absolute paths. The above configuration can also be written as:

snapshot_dir /mnt/btr_pool/btrbk_snapshots
subvolume    /mnt/btr_pool/home

If you don't want to mount the btrfs root filesystem to /mnt/btr_pool, you might as well configure it like this:

snapshot_dir /btrbk_snapshots
subvolume    /home

Start a dry run (-n, --dry-run):

# btrbk run -n

Create the first snapshot:

# btrbk run

Print schedule (-S, --print-schedule):

# btrbk run -n -S

If it works as expected, configure a cron job to run btrbk hourly:

/etc/cron.hourly/btrbk:

#!/bin/sh
exec /usr/bin/btrbk -q run

Snapshots will now be created every hour. All snapshots are preserved for at least 18 hours (snapshot_preserve_min), whether they are created by the cron job or manually by calling sudo btrbk run on the command line. Additionally, 48 hourly snapshots are preserved (snapshot_preserve).

Example: Backups to USB Disk

In this example, we assume you have a laptop with:

a disk having a btrfs root subvolume (subvolid=5) mounted on /mnt/btr_pool, containing a subvolume rootfs for the root filesystem (i.e. mounted on /) and a subvolume home for the user data,
a directory or subvolume /mnt/btr_pool/btrbk_snapshots which will hold the btrbk snapshots,
a backup disk having a btrfs volume mounted as /mnt/btr_backup, containing a subvolume or directory mylaptop for the incremental backups.

Retention policy:

keep all snapshots for 2 days, no matter how frequently you (or your cron job) run btrbk
keep daily snapshots for 14 days (very handy if you are on the road and the backup disk is not attached)
keep monthly backups forever
keep weekly backups for 10 weeks
keep daily backups for 20 days

/etc/btrbk/btrbk-mylaptop.conf:

snapshot_preserve_min   2d
snapshot_preserve      14d

# Create snapshots only if the backup disk is attached
#snapshot_create ondemand

target_preserve_min    no
target_preserve        20d 10w *m

snapshot_dir           btrbk_snapshots

volume /mnt/btr_pool
  target /mnt/btr_backup/mylaptop
  subvolume rootfs
  subvolume home
  [...]

/etc/cron.daily/btrbk:

#!/bin/sh
exec /usr/bin/btrbk -q -c /etc/btrbk/btrbk-mylaptop.conf run

This will create snapshots on a daily basis:
- /mnt/btr_pool/btrbk_snapshots/rootfs.YYYYMMDD
- /mnt/btr_pool/btrbk_snapshots/home.YYYYMMDD
And create incremental backups in:
- /mnt/btr_backup/mylaptop/rootfs.YYYYMMDD
- /mnt/btr_backup/mylaptop/home.YYYYMMDD

If you prefer triggering the backups manually, change the cron command to run the snapshot action instead of run. Start the backups manually by running:

# btrbk resume

For a quick additional snapshot of your home, run:

# btrbk snapshot home

Example: Host-initiated Backup on Fileserver

Let's say you have a fileserver at "myserver.example.org" where you want to create backups of your laptop disk. The config could look like this:

ssh_identity /etc/btrbk/ssh/id_rsa

volume /mnt/btr_pool
  subvolume rootfs
    target /mnt/btr_backup/mylaptop
    target ssh://myserver.example.org/mnt/btr_backup/mylaptop

In addition to the backups on your local usb-disk mounted at /mnt/btr_backup/mylaptop, incremental backups would also be pushed to myserver.example.org.

Example: Fileserver-initiated Backups from Several Hosts

If you're a sysadmin and want to trigger backups directly from your fileserver, the config would be something like:

ssh_identity /etc/btrbk/ssh/id_rsa

volume ssh://alpha.example.org/mnt/btr_pool
  target /mnt/btr_backup/alpha
  subvolume rootfs
  subvolume home

volume ssh://beta.example.org/mnt/btr_pool
  target /mnt/btr_backup/beta
  subvolume rootfs
  subvolume dbdata

This will pull backups from alpha/beta.example.org and locally create:

/mnt/btr_backup/alpha/rootfs.YYYYMMDD
/mnt/btr_backup/alpha/home.YYYYMMDD
/mnt/btr_backup/beta/rootfs.YYYYMMDD
/mnt/btr_backup/beta/dbdata.YYYYMMDD

Example: Multiple Btrbk Instances

Let's say we have a host (at 192.168.0.42) running btrbk with the setup of the time-machine example above, and we need a backup server to only fetch the snapshots.

/etc/btrbk/btrbk.conf (on backup server):

target_preserve_min        no
target_preserve            0d 10w *m

volume ssh://192.168.0.42/mnt/btr_pool
  target /mnt/btr_backup/my-laptop
  subvolume home
    snapshot_dir           btrbk_snapshots
    snapshot_preserve_min  all
    snapshot_create        no

If the server runs btrbk with this config, 10 weeklies and all monthlies are received from 192.168.0.42. The source filesystem is never altered because of snapshot_preserve_min all.

Btrbk

Install / Use

README