bup: It backs things up

bup is a program that backs things up. It's short for "backup." Can you believe that nobody else has named an open source program "bup" after all this time? Me neither.

Despite its unassuming name, bup is pretty cool. To give you an idea of just how cool it is, I wrote you this poem:

                         Bup is teh awesome
                      What rhymes with awesome?
                        I guess maybe possum
                       But that's irrelevant.
		

Hmm. Did that help? Maybe prose is more useful after all.

Reasons bup is awesome

bup has a few advantages over other backup software:

• It uses a rolling checksum algorithm (similar to rsync) to split large files into chunks. The most useful result of this is you can backup huge virtual machine (VM) disk images, databases, and XML files incrementally, even though they're typically all in one huge file, and not use tons of disk space for multiple versions.

• It uses the packfile format from git (the open source version control system), so you can access the stored data even if you don't like bup's user interface.

• Unlike git, it writes packfiles directly (instead of having a separate garbage collection / repacking stage) so it's fast even with gratuitously huge amounts of data. bup's improved index formats also allow you to track far more filenames than git (millions) and keep track of far more objects (hundreds or thousands of gigabytes).

• Data is "automagically" shared between incremental backups without having to know which backup is based on which other one - even if the backups are made from two different computers that don't even know about each other. You just tell bup to back stuff up, and it saves only the minimum amount of data needed.

• You can back up directly to a remote bup server, without needing tons of temporary disk space on the computer being backed up. And if your backup is interrupted halfway through, the next run will pick up where you left off. And it's easy to set up a bup server: just install bup on any machine where you have ssh access.

• Bup can use "par2" redundancy to recover corrupted backups even if your disk has undetected bad sectors.

• Even when a backup is incremental, you don't have to worry about restoring the full backup, then each of the incrementals in turn; an incremental backup acts as if it's a full backup, it just takes less disk space.

• You can mount your bup repository as a FUSE filesystem and access the content that way, and even export it over Samba.

• It's written in python (with some C parts to make it faster) so it's easy for you to extend and maintain.

Reasons you might want to avoid bup

• It's not remotely as well tested as something like tar, so it's more likely to eat your data. It's also missing some probably-critical features, though fewer than it used to be.

• While it is intended to work with python 3.7 or newer, a C compiler, and an installed git version >= 1.7.2, it is currently only automatically tested against some python versions 3.9 and newer and git versions 2.3 and newer. Please report any discrepancies. It also requires par2 if you want fsck to be able to generate the information needed to recover from some types of corruption.

• It has only been reported to work on Linux, FreeBSD, NetBSD, OS X

  = 10.4, Solaris, or Windows (with Cygwin, and WSL). Patches to support other platforms are welcome.

• Any items in "Things that are stupid" below.

Notable changes introduced by a release

• <a href="note/0.33.10-from-0.33.9.md">Changes in 0.33.10 as compared to 0.33.9</a>
• <a href="note/0.33.9-from-0.33.8.md">Changes in 0.33.9 as compared to 0.33.8</a>
• <a href="note/0.33.8-from-0.33.7.md">Changes in 0.33.8 as compared to 0.33.7</a>
• <a href="note/0.33.7-from-0.33.6.md">Changes in 0.33.7 as compared to 0.33.6</a>
• <a href="note/0.33.6-from-0.33.5.md">Changes in 0.33.6 as compared to 0.33.5</a>
• <a href="note/0.33.5-from-0.33.4.md">Changes in 0.33.5 as compared to 0.33.4</a>
• <a href="note/0.33.4-from-0.33.3.md">Changes in 0.33.4 as compared to 0.33.3</a>
• <a href="note/0.33.3-from-0.33.2.md">Changes in 0.33.3 as compared to 0.33.2</a>
• <a href="note/0.33.2-from-0.33.1.md">Changes in 0.33.2 as compared to 0.33.1</a>
• <a href="note/0.33.1-from-0.33.md">Changes in 0.33.1 as compared to 0.33</a>
• <a href="note/0.33-from-0.32.md">Changes in 0.33 as compared to 0.32</a>
• <a href="note/0.32.1-from-0.32.md">Changes in 0.32.1 as compared to 0.32</a>
• <a href="note/0.32-from-0.31.md">Changes in 0.32 as compared to 0.31</a>
• <a href="note/0.31-from-0.30.1.md">Changes in 0.31 as compared to 0.30.1</a>
• <a href="note/0.30.1-from-0.30.md">Changes in 0.30.1 as compared to 0.30</a>
• <a href="note/0.30-from-0.29.3.md">Changes in 0.30 as compared to 0.29.3</a>
• <a href="note/0.29.3-from-0.29.2.md">Changes in 0.29.3 as compared to 0.29.2</a>
• <a href="note/0.29.2-from-0.29.1.md">Changes in 0.29.2 as compared to 0.29.1</a>
• <a href="note/0.29.1-from-0.29.md">Changes in 0.29.1 as compared to 0.29</a>
• <a href="note/0.29-from-0.28.1.md">Changes in 0.29 as compared to 0.28.1</a>
• <a href="note/0.28.1-from-0.28.md">Changes in 0.28.1 as compared to 0.28</a>
• <a href="note/0.28-from-0.27.1.md">Changes in 0.28 as compared to 0.27.1</a>
• <a href="note/0.27.1-from-0.27.md">Changes in 0.27.1 as compared to 0.27</a>

Test status

| main | |--------| | |

Getting started

From source

• Check out the bup source code using git (for Cygwin, use a Cygwin installed git so that symlinks will work by defaul):

  git clone https://github.com/bup/bup
  

• This will leave you on the main branch, which is perfect if you would like to help with development, but if you'd just like to use bup, please check out the latest stable release like this:

  git checkout 0.33.10
  

  You can see the latest stable release here: https://github.com/bup/bup/tags

• Ensure you have a Python 3.7+ development environment available and install the required python libraries (including the development libraries).

  See the relevant platform specific information below, if available, and if not, refer to the Debian notes to get a general idea of what's required. You might also want to look at the dev/prep-for-* scripts. Those include all the commands used to prepare to build bup for testing on the platform.

  For bup fuse you will need to install python-fuse rather than fusepy. For example, in Debian, install python3-fuse rather than python3-fusepy.

  If you would like to use the optional bup web server on systems without a tornado package, you may want to try this:

  pip install tornado
  

• Build (below, replace make with whatever GNU make is called on the current system if it isn't make, e.g. commonly gmake):

  make
  

  At the moment the build treats compiler warnings as errors. If the build fails as a result, try this:

  CFLAGS=-Wno-error ./configure
  make
  

• Run the tests:

  make long-check
  

  or if you're in a bit more of a hurry:

  make check
  

  Additional (and some slightly differing) tests will be run if you're either root, or running under fakeroot, and for the root case, in order to include all of the tests, you'll want to make sure the fuse and loop modules are loaded, which you can do for Linux via (as root):

  modprobe fuse
  modprobe loop
  

  If you have the Python xdist module installed, then you can probably run the tests faster by adding the make -j N option, with an N of perhaps one more than your CPU core count (see <a href="HACKING">./HACKING</a> for additional information):

  make -j 5 check
  

  The tests should pass (with some skipped tests that weren't applicable in your environment). If they don't pass for you, stop here and send an email to bup-list@googlegroups.com. Though if there are symbolic links along the current working directory path, the tests may fail. Running something like this before "make test" should sidestep the problem:

  cd "$(pwd -P)"
  

• You can install bup via "make install", and override the default destination with DESTDIR and PREFIX.

  Files are normally installed to "$DESTDIR/$PREFIX" where DESTDIR is empty by default, and PREFIX is set to /usr/local. So if you wanted to install bup to /opt/bup, you might do something like this:

  make install DESTDIR=/opt/bup PREFIX=''
  

• The Python version that bup will use is determined by the python-config program chosen by ./configure, which will search for a reasonable version unless BUP_PYTHON_CONFIG is set in the environment. You can see which Python executable was chosen by looking at the configure output, or examining config/config.var/bup-python-config, and you can change the selection by re-running ./configure.

• If you want to specify your own CPPFLAGS, CFLAGS, or LDFLAGS, you can set them for individual make invocations, e.g. make CFLAGS=-O0 check, or persistently via ./configure with CFLAGS=-O0 ./configure. At the moment, make clean clears the configuration, b