Filemap

Usage: fm command args... For more help, specify one of the following commands followed by -h option: cat - Concatenate one or more files to stdout chgrp - Change GID on files in the cloud chmod - Change permissions on files in the cloud df - Show free space on each node fs - None get - Copy one or more files from the cloud to local storage init - Create/correct directory structure jobs - Show all of the jobs still active kill - Kill a job loadavg - Show load average ls - File & directory listing map - Launch a computation on a set of input files in the cloud mkdir - Make a directory (or directories) on all nodes mv - Rename files in the cloud partition - Partition an input file plot - Plot run-time of a previous map replicate - Replicate the specified files (in the virtual file system) from each node rm - Delete a file or directory tree in the cloud run - Run a command on each node and print results runhost - Run a command on each unique host and print results store - Store one or more files into the cloud

Options: -h, --help show this help message and exit -v, --verbose Increase verbosity (can be used multiple times)

FileMap is a file-based map-reduce system.
You can think of it is parallel and distributed xargs or make.
It features replicated storage and intermediate result caching.

https://github.com/mfisk/filemap/wiki