clsync

0 - Contents

1. Name
2. Motivation
3. inotify vs fanotify
4. Installing
5. How to use
6. Example of usage
7. Other uses
8. Clustering
9. FreeBSD support
10. Support
11. Developing
12. Articles
13. See also

1 - Name

Why clsync? The first name of the utility was insync (due to inotify) but then I was suggested to use fanotify instead of inotify and utility has been renamed to fasync. Then I started to intensively write the program and I faced with some problems in fanotify (see "inotify vs fanotify"). So I was have to temporary fallback to inotify, so I decided that the best name is "Runtime Sync" or "Live Sync" but rtsync is a name of some corporation and lsync is busy by "lsyncd". So I called it clsync that should be interpreted as "lsync but on c" due to "lsyncd" that written on "LUA" and may be used for similar purposes.

2 - Motivation

This utility has been written for two purposes:

• for making high availability clusters
• for making backups of them

To do a HA cluster I've tried a lot of different solutions, like "simple rsync by cron", "glusterfs", "ocfs2 over drbd", "shared replicated external storage", "incron + perl + rsync", "inosync", "lsyncd" and so on. When I started to write the utility we were using "lsyncd", "ceph" and "ocfs2 over drbd". However all of this solutions doesn't satisfy me, so I was have to write own utility for this purpose.

To do backups we also tried a lot of different solution, and again I was have to write own utility for this purpose.

The best known (for me) replacement for this utility is "lsyncd", however:

• It's code is >½ on LUA. There a lot of problems connected with it, for example:
  • It's more difficult to maintain the code with ordinary sysadmin.
  • It really eats 100% CPU sometimes.
  • It requires LUA libs, that cannot be easily installed to few of our systems.
• It's a little buggy (it crashed on our cases).
• Sometimes, it's too complex in configuration for our situation (not flexible enough).
• It doesn't support kqueue/bsm (we also had a FreeBSD-based system). etc

Long story short: "lsyncd" - is a good and useful utility, just did not fit to our needs well enough. And we spent enough much time on tuning "lsyncd" to realize that we could've already write a new tool specialized to our tasks. So there it is :)

Also clsync had been used for some other tiny tasks, like to replace incron/csync2/etc in our HPC-clusters for syncing /etc/{passwd,shadow,group,shells} files and running post-scripts.

3 - inotify vs fanotify

It's said that fanotify is much better than inotify. So I started to write this program with using of fanotify. However I encountered the problem, that fanotify was unable to catch some important events at the moment of writing the program, like "directory creation" or "file deletion". So I switched to "inotify", leaving the code for "fanotify" in the safety... So, don't use "fanotify" in this utility ;).

UPD: Starting with kernels 5.1 we will be able to use fanotify for all events ;)

4 - Installing

Linux Distributions

Some distributions already have clsync supported in the main repo:

Debian/Ubuntu:

apt-get install clsync

An optional clsync socket monitoring and control library is available in the libclsync0 package and its devel files are in the libclsync-dev

Gentoo:

emerge clsync

You may customize all clsync features via a multitude of USE flags.

Alt Linux:

apt-get install clsync

An optional clsync socket monitoring and control library is available in the libclsync package and its devel files are in the libclsync-devel. Examples are located in the clsync-examples package and doxygen API documentation is in clsync-apidocs.

From the Source Code

If it's required to install clsync from the source, first of all, you should install dependencies to compile it. Names may vary in various distributions, but you'll get the idea:

Only the following packages are mandatory: glib2-devel autoreconf gcc

Dependencies for optional features:

• libcap-devel — capabilities support for privilege separation
• libcgroup-devel — cgroups support for privilege separation
• libmhash-devel — use mhash for faster Adler-32 implementation (used only in cluster and kqueue code)
• doxygen — to build API documentation
• graphviz — to build API documentation

Next step is generating Makefile. To do that usually it's enough to execute:

autoreconf -i && ./configure

You may be interested in various configuration options, so see for details:

./configure --help

Next step is compiling. To compile usually it's enough to execute:

make -j$(nproc)

Next step is installing. To install usually it's enough to execute:

su -c 'make install'

Portable binary

It is also possible to build a portable static binary:

./configure --without-libcgroup --without-gio --disable-shared
make clean all -j$(nproc) LDFLAGS='-all-static'
ldd ./clsync

5 - How to use

How to use is described in "man" ;). What is not described, you can ask me personally (see "Support").

See also section 7 of this document.

6 - An example from scratch

Example of usage, that works on my PC is in directory "examples". Just run "clsync-start-rsyncdirect.sh" and try to create/modify/delete files/dirs in "example/testdir/from". All modifications should appear (with some delay) in directory "example/testdir/to" ;)

For dummies:

pushd /tmp
git clone https://github.com/clsync/clsync
cd clsync
autoreconf -fi
./configure
make
export PATH_OLD="$PATH"
export PATH="$(pwd):$PATH"
cd examples
./clsync-start-rsyncdirect.sh
export PATH="$PATH_OLD"

Now you can try to make changes in directory "/tmp/clsync/examples/testdir/from" (in another terminal). Wait about 7 seconds after the changes and check directory "/tmp/clsync/examples/testdir/to". To finish the experiment press ^C (control+c) in clsync's terminal.

cd ../..
rm -rf clsync
popd

Note: There's no need to change PATH's value if clsync is installed system-wide, e.g. with

make install

For dummies, again (with "make install"):

pushd /tmp
git clone https://github.com/clsync/clsync
cd clsync
autoreconf -fi
./configure
make
sudo make install
cd examples
./clsync-start-rsyncdirect.sh

Directory "/tmp/clsync/examples/testdir/from" is now synced to "/tmp/clsync/examples/testdir/to" with 7 seconds delay. To terminate the clsync press ^C (control+c) in clsync's terminal.

cd ..
sudo make uninstall
cd ..
rm -rf clsync
popd

For really dummies or/and lazy users, there's a video demonstration: http://ut.mephi.ru/oss/clsync

7 - More examples (use cases)

Mirroring a directory:

clsync -Mrsyncdirect -W/path/to/source_dir -D/path/to/destination_dir

Syncing authorized_keys files:

mkdir -p /etc/clsync/rules
printf "+w^$\n+w^[^/]+$\n+W^[^/]+/.ssh$\n+f^[^/]+/.ssh/authorized_keys$\n-*" > /etc/clsync/rules/authorized_files_only
clsync  -Mdirect -Scp -W/mnt/master/home/ -D/home -R/etc/clsync/rules/authorized_files_only -- -Pfp --parents %INCLUDE-LIST% %destination-dir%

Mirroring a directory, but faster:

clsync -w5 -t5 -T5 -Mrsyncdirect -W/path/to/source_dir -D/path/to/destination_dir

Instant mirroring of a directory:

clsync -w0 -t0 -T0 -Mrsyncdirect -W/path/to/source_dir -D/path/to/destination_dir

Making two directories synchronous:

clsync -Mrsyncdirect --background -z /var/run/clsync0.pid --output syslog -Mrsyncdirect -W/path/to/dir1 -D/path/to/dir2 --modification-signature '*'
clsync -Mrsyncdirect --background -z /var/run/clsync1.pid --output syslog -Mrsyncdirect -W/path/to/dir2 -D/path/to/dir1 --modification-signature '*'

Fixing privileges of a web-site:

clsync -w3 -t3 -T3 -x1 -W/var/www/site.example.org/root -Mdirect -Schown --uid  0  --gid  0  -Ysyslog  -b1  --modification-signature uid,gid -- --from=root www-data:www-data %INCLUDE-LIST%

'Atomic' sync:

clsync --exit-on-no-events --max-iterations=20 --mode=rsyncdirect -W/var/www_new -Srsync -- %RSYNC-ARGS% /var/www_new/ /var/www/

Moving a web-server:

clsync  --exit-on-no-events  --max-iterations=20 --pre-exit-hook=/root/stop-here.sh --exit-hook=/root/start-there.sh --mode=rsyncdirect --ignore-exitcode=23,24 --retries=3 -W /var/www -S rsync -- %RSYNC-ARGS% /var/www/ rsync://clsync@another-host/var/www/

Copying files to slave-nodes using pdcp(1):

clsync -Msimple -S pdcp -W /opt/global -b -Y syslog -- -a %INCLUDE-LIST% %INCLUDE-LIST%

Copying files to slave-nodes using uftp(1):

clsync -Mdirect -S uftp -W/opt/global --background=1 --output=syslog -- -M 248.225.233.1 %INCLUDE-LIST%

A dry running to see rsync(1) arguments that clsync will use:

clsync -Mrsyncdirect -S echo -W/path/to/source_dir -D/path/to/destination_dir

An another dry running to look how clsync will call pdcp(1):

clsync -Msimple -S echo -W /opt/global -b0 -- pdcp -a %INCLUDE-LIST% %INCLUDE-LIST%

Automatically run make build if any *.c file changed

printf "%s\n" "+f.c$" "-f" | clsync --have-recursive-sync -W . -R /dev/stdin -Mdirect -r1 --ignore-failures -t1 -w1 -Smake -- build

8 - Clustering

I've started to implement support of bi-directional syncing with using multicast notifing of other nodes. However it became a long task, so it was suspended for next releases.

However let's solve next hypothe