Repogen - Universal Repository Generator

⚠️ Alpha Software Warning

Repogen is alpha software and has not been extensively battle-tested in production environments. While it includes comprehensive test coverage and has been validated with package managers, use it with caution for critical infrastructure. Always verify generated repositories work correctly with your package manager before deploying to production.

Repogen is a CLI tool that generates static repository structures for multiple package managers. It scans directories for packages, generates appropriate metadata files, and signs repositories with GPG/RSA keys.

Supported Package Types

• Debian/APT (.deb packages)
• Yum/RPM (.rpm packages)
• Alpine/APK (.apk packages)
• Arch Linux/Pacman (.pkg.tar.zst, .pkg.tar.xz, .pkg.tar.gz)
• Homebrew (bottle files)

Features

• Automatic Package Detection: Scans directories and auto-detects package types using magic bytes
• Metadata Generation: Creates all necessary index and metadata files for each repository type
• Repository Signing: Signs repositories with GPG (Debian/RPM/Pacman) or RSA (Alpine) keys
• Unsigned Repository Support:
  • Always generates InRelease files (required by Debian Trixie)
  • InRelease contains Release content without signature for unsigned repos
  • Compatible with [trusted=yes] apt option
• Static Output: Generates static file structures that can be served by any web server
• Simple Component Structure: Uses single component/pool structure for simplicity

Installation

From Source

# Clone the repository
git clone https://github.com/ralt/repogen
cd repogen

# Build
go build -o repogen ./cmd/repogen

# Optional: Install to PATH
sudo cp repogen /usr/local/bin/

Prerequisites

• Go 1.23 or later

Usage

Basic Usage

# Scan current directory and generate repositories
repogen generate

# Scan specific directory
repogen generate --input-dir /path/to/packages --output-dir /path/to/repo

# Enable verbose logging
repogen generate -v

Incremental Mode

Incremental mode allows you to add new packages to an existing repository without regenerating everything from scratch. This is useful when:

• You have a large repository and only want to add new package versions
• You're syncing from S3 and don't want to download all package files locally
• You want faster repository updates

How It Works:

1. Reads existing metadata files (Packages, trust.db, repomd.xml, etc.) from the output directory
2. Adds only new packages without removing existing ones
3. Errors if a package with the same name+version already exists (conflict detection)
4. Regenerates metadata files with both existing and new packages
5. Re-signs metadata if signing is enabled

Basic Incremental Usage:

# Add new packages to existing repository
repogen generate \
  --input-dir ./new-packages \
  --output-dir ./repo \
  --incremental

S3 Workflow Examples:

The incremental mode is particularly powerful when combined with S3. You can sync only the metadata files (not the packages themselves), add new packages, and regenerate.

Debian Repository

# Sync only metadata from S3 (not package files)
aws s3 sync s3://my-bucket/repo/dists ./repo/dists --delete

# Add new packages with repogen
repogen generate --input-dir ./new-packages --output-dir ./repo --incremental

# Sync everything back to S3 (without --delete to preserve existing packages)
aws s3 sync ./repo s3://my-bucket/repo

RPM Repository

# Sync only metadata
aws s3 sync s3://my-bucket/repo/40/x86_64/repodata ./repo/40/x86_64/repodata --delete

# Add new packages
repogen generate \
  --input-dir ./new-packages \
  --output-dir ./repo \
  --incremental \
  --version 40

# Sync back (without --delete to preserve existing packages)
aws s3 sync ./repo s3://my-bucket/repo

Pacman Repository

# Sync only database files (exclude actual package files)
aws s3 sync s3://my-bucket/repo/x86_64 ./repo/x86_64 \
  --exclude "*.pkg.tar.zst" \
  --exclude "*.pkg.tar.zst.sig"

# Add new packages
repogen generate \
  --input-dir ./new-packages \
  --output-dir ./repo \
  --repo-name myrepo \
  --incremental

# Sync back (without --delete to preserve existing packages)
aws s3 sync ./repo s3://my-bucket/repo

APK Repository

# Sync metadata only
aws s3 cp s3://my-bucket/repo/x86_64/APKINDEX.tar.gz ./repo/x86_64/APKINDEX.tar.gz

# Add new packages
repogen generate --input-dir ./new-packages --output-dir ./repo --incremental

# Sync back (without --delete to preserve existing packages)
aws s3 sync ./repo s3://my-bucket/repo

Important Notes:

• Incremental mode will error if a package with the same name+version already exists (conflict detection)
• If metadata files don't exist, it falls back to normal mode automatically
• Package files from existing metadata don't need to be present locally
• You can use incremental mode with or without signing

With Signing

Debian/RPM/Pacman (GPG Signing)

# Generate signed Debian/RPM repositories
repogen generate \
  --input-dir ./packages \
  --output-dir ./repo \
  --gpg-key /path/to/private.key \
  --gpg-passphrase "your-passphrase"

# Generate signed Pacman repository (requires --repo-name)
repogen generate \
  --input-dir ./packages \
  --output-dir ./repo \
  --repo-name "myrepo" \
  --gpg-key /path/to/private.key \
  --gpg-passphrase "your-passphrase"

Alpine (RSA Signing)

# Generate signed Alpine repository
repogen generate \
  --input-dir ./packages \
  --output-dir ./repo \
  --rsa-key /path/to/rsa-private.pem \
  --rsa-passphrase "your-passphrase" \
  --key-name "mykey"

Configuration Options

repogen generate [flags]

Flags:
  # Input/Output
  -i, --input-dir string        Input directory to scan (default ".")
  -o, --output-dir string       Output directory (default "./repo")
  -v, --verbose                 Enable verbose logging

  # Incremental Mode
      --incremental             Add new packages to existing repository without removing existing ones

  # GPG Signing (Debian/RPM)
  -k, --gpg-key string          Path to GPG private key
  -p, --gpg-passphrase string   GPG key passphrase

  # RSA Signing (Alpine)
      --rsa-key string          Path to RSA private key
      --rsa-passphrase string   RSA key passphrase
      --key-name string         Key name for Alpine signatures (default "repogen")

  # Repository Metadata
      --origin string           Repository origin name
      --label string            Repository label
      --repo-name string        Repository name (required for Pacman)
      --codename string         Codename for Debian repos (default "stable")
      --suite string            Suite for Debian repos (defaults to codename)
      --components strings      Components for Debian repos (default [main])
      --arch strings            Architectures to support (default [amd64])

  # Homebrew
      --base-url string         Base URL for Homebrew bottles

Generated Repository Structures

Debian/APT Repository

repo/
├── dists/
│   └── stable/
│       ├── InRelease              # Cleartext signed Release (or unsigned copy for unsigned repos)
│       ├── Release                # Main metadata
│       ├── Release.gpg            # Detached GPG signature (only for signed repos)
│       └── main/
│           └── binary-amd64/
│               ├── Packages        # Package metadata
│               ├── Packages.gz     # Compressed
│               └── Release
└── pool/
    └── main/
        └── {letter}/              # First letter of package name
            └── {package-name}/
                └── package.deb

Using the Repository:

# Add repository (unsigned)
echo "deb [trusted=yes] http://your-server.com/repo stable main" | sudo tee /etc/apt/sources.list.d/repo.list

# Add repository (signed)
# First, import the public key
wget -qO - http://your-server.com/repo/public.key | sudo apt-key add -
echo "deb http://your-server.com/repo stable main" | sudo tee /etc/apt/sources.list.d/repo.list

# Update and install
sudo apt update
sudo apt install package-name

RPM/Yum Repository

repo/
├── repodata/
│   ├── repomd.xml              # Main metadata index
│   ├── repomd.xml.asc          # GPG signature
│   └── {hash}-primary.xml.gz   # Package metadata
└── Packages/
    └── *.rpm

Using the Repository:

# Create repo file
sudo tee /etc/yum.repos.d/repo.repo <<EOF
[myrepo]
name=My Repository
baseurl=http://your-server.com/repo
enabled=1
gpgcheck=0
EOF

# With GPG checking
sudo rpm --import http://your-server.com/repo/public.key
sudo tee /etc/yum.repos.d/repo.repo <<EOF
[myrepo]
name=My Repository
baseurl=http://your-server.com/repo
enabled=1
gpgcheck=1
gpgkey=http://your-server.com/repo/public.key
EOF

# Install packages
sudo yum install package-name

Alpine/APK Repository

repo/
└── x86_64/
    ├── APKINDEX.tar.gz         # Package index
    ├── APKINDEX.tar.gz.SIGN.RSA.repogen.pub  # RSA signature
    └── package-1.0.0-r0.apk

Using the Repository:

# Add repository
echo "http://your-server.com/repo" | sudo tee -a /etc/apk/repositories

# With signing (copy public key first)
sudo cp repogen.pub /etc/apk/keys/
echo "http://your-server.com/repo" | sudo tee -a /etc/apk/repositories

# Update and install
sudo apk update
sudo apk add package-name

Arch Linux/Pacman Repository

repo/
└── x86_64/
    ├── myrepo.db.tar.zst       # Package database
    ├── myrepo.db               # Symlink/copy of .db.tar.zst
    ├── myrepo.db.tar.zst.sig   # GPG signature (if signed)
    ├── myrepo.db.sig           # Symlink/copy of signature
    ├── package-1.0.0-1-x86_64.pkg.tar.zst
    └── p