GitHub to Gitea Mirror

This tool sets up and manages pull mirrors from GitHub repositories to Gitea repositories, including the entire codebase, issues, PRs, releases, and wikis.

I've been eagerly awaiting Gitea's PR 20311 for over a year, but since it keeps getting pushed out for every release I figured I'd create something in the meantime.

Blunt disclaimer: This is a hobby project, and I hope the PR above will be merged and implemented soon. When it is, this project will have served its purpose. I created it from scratch using Cursor and Claude 3.7 Sonnet.

Features

• Web UI for managing mirrors and viewing logs screens and more info
• Set up GitHub repositories as pull mirrors in Gitea
• Mirror the entire codebase, issues, and PRs (not just releases)
• Mirror GitHub releases and release assets with full descriptions and attachments
• Mirror GitHub wikis to separate Gitea repositories
• Auto-discover mirrored repositories in Gitea
• Support for both full GitHub URLs and owner/repo format
• Comprehensive logging with direct links to logs from error messages
• Support for large release assets with dynamic timeouts
• Asset download and upload with size-based timeout calculation
• Scheduled mirroring with configurable interval
• Enhanced UI with checkboxes for configuration options
• Dark mode support
• Error handling and visibility

Quick Start

Get up and running in minutes:

# Clone the repository
git clone https://github.com/jonasrosland/gitmirror.git
cd gitmirror

# Copy and configure the example .env file
cp .env.example .env
# Edit the .env file with your tokens and Gitea URL

# Start the application
docker-compose up -d

# Access the web UI
# Open http://localhost:5000 in your browser

Prerequisites

• Docker and Docker Compose (for running the application)
• GitHub Personal Access Token with repo scope
• Gitea Access Token with read:user, write:repository, and write:issue scopes
• Access to both GitHub and Gitea repositories

Configuration

Create a .env file in the same directory as the docker-compose.yml with the following variables:

# GitHub Personal Access Token (create one at https://github.com/settings/tokens)
# Required scopes: repo (for private repositories)
# For public repositories, this is optional but recommended
GITHUB_TOKEN=your_github_token

# Gitea Access Token (create one in your Gitea instance under Settings > Applications)
# Required permissions: read:user, write:repository, write:issue
GITEA_TOKEN=your_gitea_token

# Your Gitea instance URL (no trailing slash)
GITEA_URL=https://your-gitea-instance.com

# Secret key for the web UI (OPTIONAL)
# This key is used to secure Flask sessions and flash messages
# If not provided, a random key will be automatically generated at container start
# SECRET_KEY=your_secret_key

# Log retention days (OPTIONAL, defaults to 30 days)
LOG_RETENTION_DAYS=30

Authentication for Private Repositories

If you want to mirror private GitHub repositories, you must provide a GitHub token with the repo scope. This token is used to authenticate with GitHub when creating the mirror.

For public repositories, the GitHub token is optional but recommended to avoid rate limiting issues.

Usage

Using Docker Compose (Recommended)

For easier deployment, you can use Docker Compose:

1. Start the web UI:

docker-compose up -d

2. Run the mirror script (one-time execution):

docker-compose run --rm mirror

3. Run the mirror script for a specific repository:

docker-compose run --rm mirror mirror owner/repo gitea_owner gitea_repo

4. Run with specific flags:

# Enable mirroring metadata (issues, PRs, labels, milestones, wikis)
docker-compose run --rm mirror mirror --mirror-metadata

# Force recreation of empty repositories (required when an existing repository is empty but not a mirror)
docker-compose run --rm mirror mirror --force-recreate

# Combine flags for a specific repository
docker-compose run --rm mirror mirror owner/repo gitea_owner gitea_repo --mirror-metadata --force-recreate

5. View logs:

docker-compose logs -f

6. Stop the services:

docker-compose down

Using Bind Mounts for Logs

By default, logs are stored in a Docker volume for better permission handling. If you prefer to use bind mounts instead (to access logs directly on the host filesystem), you can modify the docker-compose.yml file:

1. Change the volume configuration from:

volumes:
  - gitmirror_logs:/app/logs

to:

volumes:
  - ./logs:/app/logs

2. Create the logs directory with the correct permissions:

mkdir -p logs
chmod 755 logs

3. Update the container user to match your host user's UID/GID:

environment:
  - PUID=$(id -u)
  - PGID=$(id -g)
user: "${PUID}:${PGID}"

4. Remove the volumes section at the bottom of the file that defines gitmirror_logs

This setup will:

• Store logs directly in the logs directory on your host
• Allow you to access logs without using Docker commands
• Maintain proper permissions for the container to write logs
• Keep the same log rotation and retention settings

Using Docker Directly

To run the application with Docker directly:

1. Build the Docker image:

docker build -t github-gitea-mirror .

2. Run the container:

   a. Run the web UI (default mode):

   docker run --rm -p 5000:5000 --env-file .env github-gitea-mirror
   

   b. Run the mirror script in auto-discovery mode:

   docker run --rm --env-file .env github-gitea-mirror mirror
   

   c. Run the mirror script for a specific repository:

   docker run --rm --env-file .env github-gitea-mirror mirror owner/repo gitea_owner gitea_repo
   

   d. Run with the force-recreate flag (for empty repositories):

   docker run --rm --env-file .env github-gitea-mirror mirror owner/repo gitea_owner gitea_repo --force-recreate
   

3. For persistent storage of logs, mount a volume:

   docker run --rm -p 5000:5000 -v ./logs:/app/logs --env-file .env github-gitea-mirror
   

How Mirroring Works

When you set up a repository for mirroring, the script performs several types of synchronization:

1. Code Mirroring: Uses Gitea's built-in pull mirror functionality to sync:

   • The entire codebase
   • All branches and tags

   NOTE: This is done automatically at creation of the mirror repo, and sometimes it takes a while for Gitea to finish the first code sync

2. Release Mirroring: Uses custom code to sync:

   • Releases and release assets
   • Release descriptions and metadata
   • Release attachments with proper naming and descriptions

3. Metadata Mirroring: Syncs additional GitHub data:

   • Issues and their comments
   • Pull requests and their comments
   • Labels and milestones
   • Wiki content (if enabled)

The process works as follows:

1. The script checks if the repository exists in Gitea
2. If it exists and is already a mirror:
   • It triggers a code sync
   • It only mirrors releases and metadata if those options are explicitly enabled in the repository configuration
3. If it exists but is not a mirror:
   • If the target repository is empty, it requires explicit confirmation via the --force-recreate flag used with the CLI command (see below) before deleting and recreating it as a mirror
   • If the target repository has commits, it warns you that you need to to delete it manually
4. If it doesn't exist, it creates a new repository as a mirror
5. After setting up the mirror, it triggers a code sync in Gitea
6. It only mirrors releases, issues, PRs, and other metadata if those options are enabled in the repository configuration

By default, all mirroring options (metadata, releases, etc.) are disabled for safety. You can enable them through the web UI's repository configuration page or by using the appropriate command-line flags.

Repository Safety

The tool includes safety measures to prevent accidental data loss:

1. Empty Repository Protection: When an existing repository is empty but not configured as a mirror, the tool will not automatically delete and recreate it without explicit confirmation via the --force-recreate flag.

2. Non-Empty Repository Protection: If a repository contains commits, the tool will never attempt to delete it, even with the --force-recreate flag. This ensures that repositories with actual content are never accidentally deleted.

3. Explicit Confirmation: The --force-recreate flag serves as an explicit confirmation that you want to delete and recreate empty repositories as mirrors, providing an additional safety layer against accidental data loss.

4. CLI-Only Operation: The --force-recreate flag is deliberately available only through the command-line interface and not through the web UI. This design choice prevents accidental repository deletion through misclicks in the web UI and ensures that repository recreation is a deliberate, intentional action that requires specific command knowledge.

This multi-layered approach to safety ensures that repositories are protected from accidental deletion while still providing the flexibility to recreate empty repositories when necessary.

Wiki Mirroring

When mirroring a GitHub repository with a wiki, the tool creates a separate repository for the wiki content. This is necessary because:

1. Gitea's Limitations: Gitea's repository mirroring feature doesn't automatically mirror the wiki repository. Wikis in Git are actually separate repositories (with .wiki.git suffix).

2. Read-Only Constraint: For mirrored repositories in Gitea, the wiki is read-only and cannot be directly pushed to, which prevents direct mirroring of wiki content.

The mirr