WaitLock

WaitLock is a portable UNIX/POSIX command-line tool that provides mutex and semaphore functionality for shell scripts. It enables synchronized access to resources across multiple processes with automatic cleanup when processes die.

Features

• Mutex Mode: Single lock holder (default)
• Semaphore Mode: Multiple concurrent lock holders
• Automatic Cleanup: Locks released when process dies
• Signal-based Release: Clean lock release with --done flag
• CPU-aware Locking: Can scale locks to CPU count
• Lock Inspection: List and check active locks
• Multiple Output Formats: Human, CSV, and null-separated
• Command Execution: Run commands while holding locks
• UNIX Integration: Environment variables, stdin, syslog
• Portable C Implementation: Runs on any POSIX system

Quick Start

# Install dependencies (Ubuntu/Debian)
sudo apt-get install build-essential autoconf

# Build and install
./configure
make
sudo make install

# Basic usage - acquire exclusive lock (RECOMMENDED)
waitlock myapp || {
    echo "Another instance is already running"
    exit 1
}
# ... do exclusive work ...
# Lock automatically released when script exits

# Execute command with lock (BEST PRACTICE)
waitlock database_backup --exec "/usr/local/bin/backup.sh --daily"

# List active locks
waitlock --list

Table of Contents

• Installation
• Usage
• Examples
• Command Reference
• Environment Variables
• Exit Codes
• Advanced Usage
• Contributing
• License

Installation

From Source

Prerequisites

• C compiler (gcc, clang, or compatible)
• GNU Make
• autoconf (for building from git)

Build Instructions

# Clone the repository
git clone https://github.com/user/waitlock.git
cd waitlock

# Generate configure script (if building from git)
autoreconf -fi

# Configure and build
./configure
make

# Run tests
make check

# Install system-wide
sudo make install

# Or install to custom prefix
./configure --prefix=/usr/local
make install

Build Options

# Debug build
./configure CFLAGS="-g -O0 -DDEBUG"

# Release build with optimizations
./configure CFLAGS="-O2 -DNDEBUG"

# Cross-compilation example
./configure --host=arm-linux-gnueabihf

Package Installation

# Ubuntu/Debian (when available)
sudo apt-get install waitlock

# CentOS/RHEL (when available)
sudo yum install waitlock

# macOS with Homebrew (when available)
brew install waitlock

Usage

Basic Syntax

waitlock [options] <descriptor>
waitlock --list [--format=<fmt>] [--all|--stale-only]
waitlock --check <descriptor>
echo <descriptor> | waitlock [options]

Simple Examples

# Acquire mutex lock (RECOMMENDED APPROACH)
waitlock myapp || {
    echo "Another instance is already running"
    exit 1
}
# ... do exclusive work ...
# Lock automatically released when script exits

# Check if lock is available
if waitlock --check myapp; then
    echo "Lock is available"
else
    echo "Lock is held by another process"
fi

# Execute command while holding lock (BEST PRACTICE)
waitlock backup_job --exec rsync -av /source /destination

# Use with timeout
waitlock --timeout 30 critical_resource || echo "Timeout!"

⚠️ Important: Background Usage Warning

DO NOT use background execution (&) for script coordination!

# ❌ WRONG - Don't do this for script coordination:
waitlock myapp &
# This returns immediately, whether lock was acquired or not
# Both scripts may think they got the lock
# Requires complex PID management and cleanup

# ✅ CORRECT - Use foreground execution:
waitlock myapp || {
    echo "Another instance is already running"
    exit 1
}
# This blocks until lock is acquired or fails
# Clear success/failure indication
# Automatic cleanup when script exits

Why foreground is better:

• ✅ Reliable - Clear success/failure indication
• ✅ Simple - No PID management or manual cleanup needed
• ✅ Safe - No race conditions
• ✅ Automatic - Lock released when process exits

When to use background (&):

• ⚠️ Only for testing - When you need to verify lock behavior
• ⚠️ Never for production - Use --exec or foreground instead

Examples

1. Basic Mutex (Exclusive Access)

#!/bin/bash
# Ensure only one backup process runs at a time

waitlock database_backup || {
    echo "Another backup is already running"
    exit 1
}

# Perform backup
mysqldump --all-databases > backup.sql
gzip backup.sql

# Lock automatically released when script exits

2. Semaphore (Multiple Concurrent Access)

#!/bin/bash
# Allow up to 4 concurrent download processes

waitlock --allowMultiple 4 download_pool || {
    echo "Too many downloads already running"
    exit 1
}

# Perform download
wget "https://example.com/file.tar.gz"

# Lock automatically released when script exits

3. CPU-Based Semaphore

#!/bin/bash
# Use one lock per CPU core, reserving 2 cores for system

waitlock --onePerCPU --excludeCPUs 2 cpu_intensive_task || {
    echo "All CPU slots are busy"
    exit 1
}

# Run CPU-intensive task
./compute_job.sh

4. Command Execution Mode

#!/bin/bash
# Execute command while holding lock (recommended approach)

waitlock database_backup --exec bash -c "
    mysqldump --all-databases > backup.sql
    gzip backup.sql
    echo 'Backup completed'
"

5. Lock Monitoring and Management

#!/bin/bash
# Monitor active locks

# List all locks in human-readable format
waitlock --list

# List in CSV format for parsing
waitlock --list --format csv

# Show only stale locks
waitlock --list --stale-only

# Count active locks
waitlock --list --format csv | tail -n +2 | wc -l

6. Pipeline and Batch Processing

#!/bin/bash
# Process files with controlled parallelism

find /data -name "*.csv" | while read file; do
    basename "$file" | waitlock --allowMultiple 3 --exec process_file "$file"
done

# Or with xargs for better performance
find /data -name "*.csv" | \
    xargs -P 10 -I {} sh -c 'waitlock -m 3 batch_processor --exec "process_file {}"'

7. Using with Environment Variables

#!/bin/bash
# Configure via environment variables

export WAITLOCK_TIMEOUT=60
export WAITLOCK_DIR="/var/lock/myapp"
export WAITLOCK_DEBUG=1

waitlock myapp_task --syslog --syslog-facility local0

8. Error Handling

#!/bin/bash
# Robust error handling

waitlock --timeout 30 critical_resource
case $? in
    0) echo "Lock acquired successfully" ;;
    1) echo "Lock is busy" >&2; exit 1 ;;
    2) echo "Timeout expired" >&2; exit 1 ;;
    3) echo "Usage error" >&2; exit 1 ;;
    *) echo "Unexpected error" >&2; exit 1 ;;
esac

# Your critical section here
perform_critical_operation

9. Signal-based Lock Release

#!/bin/bash
# Clean lock release using --done flag (for testing/debugging)

# ⚠️ Note: This is mainly for testing - use --exec for production

# Start long-running process with lock (background for demonstration)
waitlock long_running_task &
LOCK_PID=$!

# Simulate some work
sleep 2

# Later, signal the process to release the lock cleanly
waitlock --done long_running_task

# Wait for the process to exit gracefully
wait $LOCK_PID
echo "Process exited with code: $?"

# ✅ BETTER APPROACH: Use --exec for production
# waitlock long_running_task --exec "./long_running_script.sh"

10. Resource Pool Management

#!/bin/bash
# Manage GPU resources (RECOMMENDED APPROACH)

# Acquire semaphore slot and run computation
waitlock --allowMultiple 4 gpu_pool || {
    echo "All GPU slots are busy"
    exit 1
}

# Use WAITLOCK_SLOT environment variable for GPU selection
export CUDA_VISIBLE_DEVICES=$WAITLOCK_SLOT
./gpu_computation.py

# Lock automatically released when script exits

# ✅ ALTERNATIVE: Use --exec for cleaner approach
# waitlock --allowMultiple 4 gpu_pool --exec bash -c '
#     export CUDA_VISIBLE_DEVICES=$WAITLOCK_SLOT
#     ./gpu_computation.py
# '

11. Distributed Locking (NFS)

#!/bin/bash
# Coordinate across multiple machines using NFS

export WAITLOCK_DIR="/mnt/shared/locks"

waitlock cluster_job --timeout 300 --exec bash -c "
    echo 'Running on $(hostname)'
    ./distributed_task.sh
"

Command Reference

Core Options

| Option | Description | |--------|-------------| | -m, --allowMultiple N | Allow N concurrent holders (semaphore mode) | | -c, --onePerCPU | Allow one lock per CPU core | | -x, --excludeCPUs N | Reserve N CPUs (reduce available locks by N) | | -t, --timeout SECS | Maximum wait time before giving up | | --check | Test if lock is available without acquiring | | --done | Signal lock holder to release lock (sends SIGTERM) | | -e, --exec CMD | Execute command while holding lock |

Output Options

| Option | Description | |--------|-------------| | -q, --quiet | Suppress all non-error output | | -v, --verbose | Verbose output for debugging | | -f, --format FMT | Output format: human, csv, null | | --syslog | Log operations to syslog | | --syslog-facility FAC | Syslog facility (daemon|local0-7) |

Management Options

| Option | Description | |--------|-------------| | -l, --list | List active locks and exit | | -a, --all | Include stale locks in list | | --stale-only | Show only stale locks |

Configuration Options

| Option | Description | |--------|-------------| | -d, --lock-dir DIR | Directory for lock files | | -h, --help | Show usage information | | -V, --version | Show vers