Zaloha2.sh

Zaloha2 is a small and simple directory synchronizer:

• Zaloha2 is a BASH script that uses only FIND, SORT and AWK.
• All you need is the Zaloha2.sh file (contains ~118 kB of program code and ~100 kB of documentation).
• No compilation, installation and configuration is required.
• Cyber-secure: No new binary code, no new open ports, no talking to any "uncles" on the Internet, easily reviewable.
• Three operation modes are available: the Local Mode, the Remote Source Mode and the Remote Backup Mode.
• Local Mode: Both directories are available locally (local HDD/SSD, flash drive, mounted Samba or NFS volume).
• Remote Source Mode: the source directory is on a remote source host that can be reached via SSH/SCP and the backup directory is available locally.
• Remote Backup Mode: the source directory is available locally and the backup directory is on a remote backup host that can be reached via SSH/SCP.
• Zaloha2 does not lock files while copying them. No writing on either directory may occur while Zaloha2 runs.
• Zaloha2 always copies whole files via the operating system's CP command or the SCP command (= no delta-transfer like in RSYNC).
• Zaloha2 is not limited by memory (metadata is processed as CSV files, no limits for huge directory trees).
• Zaloha2 has optional reverse-synchronization features.
• Zaloha2 can optionally compare the contents of files.
• Zaloha2 prepares scripts for case of eventual restore (can be optionally switched off to shorten the analysis phase).

Full documentation is available both online as well as inside of Zaloha2.sh.

On Linux/Unics, Zaloha2 runs natively. On Windows, Cygwin is needed.

What are synchronization programs good for ?

Read article here.

How does Zaloha2.sh work ?

Explained in five sentences (the Local Mode)

• FIND is executed to scan both directories to obtain CSV metadata about files and subdirectories.
• The CSV metadata is compared by a sequence of sorts and AWK processing steps.
• The results (= prepared synchronization actions) are presented to the user for confirmation.
• If the user confirms, the synchronization actions are executed.
• A non-interactive regime is available as well.

Explained by an Interactive Flowchart

Open Interactive JavaScript Flowchart here.

An insight into the CSV metadata is given here

Open CSV data model here.

Explained in full detail

Read the relevant sections in the Documentation.

Obtain Zaloha2.sh

The simplest way: Under the green button "<b>Code</b>" above, choose "<b>Download ZIP</b>". From the downloaded ZIP archive, extract <code>Zaloha2.sh</code> and make it executable (<code>chmod u+x Zaloha2.sh</code>).

For running the Simple Demo, extract also the scripts <code>Simple_Demo_step1/2/3/4/5/6/7.sh</code> and make them executable.

The integrity of Zaloha2.sh can be verified by its SHA-256 hash. The hash of current version is:

sha256sum Zaloha2.sh
f80879e330745063f1f36c611153a92af824da4cdf1ef851e4f9e9744861d5f5  Zaloha2.sh

Usage Examples

Basic Usage: Synchronize <code>test_source</code> to <code>test_backup</code>:

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup"

Basic usage with Colors (on terminals with ANSI escape codes):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --color

Besides the standard synchronization, if there are files in <code>test_backup</code> that are newer (younger) than the corresponding files in <code>test_source</code>, copy them back to <code>test_source</code> (Reverse-Update):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --revUp

Besides the standard synchronization, if there are files in <code>test_backup</code> that do not exist in <code>test_source</code>, and those files are newer (younger) than the last run of Zaloha2, copy them back to <code>test_source</code> (Reverse-New):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --revNew

Besides the standard synchronization, if there are files in <code>test_backup</code> that do not exist in <code>test_source</code>, irrespective of how old they are, copy them back to <code>test_source</code> (Reverse-New-All):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --revNewAll

Do Not Remove objects from <code>test_backup</code> (unless the removals are unavoidable), just add new files and subdirectories to <code>test_backup</code> (and/or update existing files in there):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --noRemove

Exclude subdirectories <code>.git</code> from the synchronization. If such subdirectories already exist in <code>test_backup</code>, they are considered obsolete (= removals are prepared):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" \
           --findSourceOps='( -type d -a -name .git ) -prune -o'

Exclude files with ending <code>.NFO</code> from the synchronization. If such files already exist in <code>test_backup</code>, they are considered obsolete (= removals are prepared):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" \
           --findSourceOps='( -type f -a -name *.NFO ) -o'

Generally Exclude files with ending <code>.NFO</code> from the synchronization (never do anything with them). Note that the option <code>--findGeneralOps</code> is used instead of <code>--findSourceOps</code>. The <code>+</code> sign tells Zaloha2 to use the passed FIND expression in addition to the internally defined defaults:

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" \
           --findGeneralOps='+( -type f -a -name *.NFO ) -o'

Disable the internally defined defaults for <code>--findGeneralOps</code> that exclude the well-known Linux and Windows "trash" and "lost+found" subdirectories: Use the option <code>--findGeneralOps</code> without the leading <code>+</code> sign to replace the internally defined defaults (here by an empty string):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" \
           --findGeneralOps=

A more complex example with several FIND expressions: Do not backup any subdirectories <code>.git</code>, <code>.svn</code> and <code>.cache</code> in <code>test_source</code>. If they already exist in <code>test_backup</code>, prepare their removals. These are also clear cases for <code>--findSourceOps</code>. Further, completely ignore (in both directories) any coredump files and the subdirectories <code>mount</code> and <code>database</code> located directly under <code>test_source</code> and <code>test_backup</code>. The reasons are that arbitrary external media can be mounted under <code>mount</code> and the backup of <code>database</code> is implemented by dedicated tools provided by the database vendor (also not on file level). These are again clear cases for <code>--findGeneralOps</code>. Additionally, preserve the internally defined defaults for <code>--findGeneralOps</code> (this is achieved by the leading <code>+</code> sign). Finally, for a good visual appearance, use the expression by expression method:

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup"      \
           --findSourceOps='( -type d -a -name .git ) -prune -o'    \
           --findSourceOps='( -type d -a -name .svn ) -prune -o'    \
           --findSourceOps='( -type d -a -name .cache ) -prune -o'  \
           --findGeneralOps='+'                                     \
           --findGeneralOps='( -type f -a -name core ) -o'          \
           --findGeneralOps='-path ///d/mount -prune -o'            \
           --findGeneralOps='-path ///d/database -prune -o'

Compare files Byte-By-Byte instead of by just their sizes and modification times (warning: this might take much time):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --byteByByte

Do not prepare scripts for the case of restore (No Restore Scripts, saves processing time and disk space):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --noRestoreScripts

Instead of GNU AWK, use MAWK, the very fast AWK implementation based on a bytecode interpreter:

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --mawk

Produce less screen output (No Progress Messages from the analysis phase):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --noProgress

Do Not Execute the actions (just prepare the scripts, see the Documentation for more details):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" --noExec

Put Zaloha2 Metadata (working files of Zaloha2.sh) to a separate storage, e.g. for speed reasons (please note that if Zaloha2.sh is used for multiple synchronizations, then each such use (instance) of Zaloha2.sh requires a separate Metadata directory):

Zaloha2.sh --sourceDir="test_source" --backupDir="test_backup" \
           --metaDir="/fast_storage/Zaloha_metadata/data_of_instance_0001"

Show the Documentation (quit viewing by typing <code>q</code>):

Zaloha2.sh --help

So much typing?

Some command lines may get lengthy. But: Usually you do not create new setups each day, nor you type the commands always from scratch. After having fine-tuned a given synchronization/backup process, you are advised to materialize the command line of Zaloha2 into a wrapper script. The wrapper script can then be attached to a desktop icon, for instance, so that the whole process can be started just by double-clicking on that icon. The advantage of wrapper scripts is that yo