HubProxy

Overview

HubProxy is a proxy for GitHub webhooks, built for people building with GitHub at scale. It fixes a lot of stuff, and makes life easier.

It has a lot of features (most optional) and it's extremely configurable.

Key Features

Webhook Verification: Cryptographically verifies GitHub webhook signatures to ensure authenticity
Event Persistence: Stores webhook events in a database (SQLite/PostgreSQL/MySQL) for audit and replay
Event Replay:
- Replay individual events by ID for testing or recovery
- Replay events within a specific time range with filtering options
Event Filtering:
- Filter events by type, repository, sender, and time range
- Query historical events through a RESTful API
- Get event statistics and delivery metrics
REST API:
- List and search webhook events with pagination
- View event type statistics over time
- Replay single events or event ranges
- Filter and query capabilities for all operations
Monitoring:
- Provides metrics and logging for webhook delivery status and performance
- Track event patterns and volume through API statistics

Why HubProxy?

Reliability:
- Never miss a webhook due to temporary service outages or bad deploys of your application
- Replay events after recovering from downtime
- Queue and retry failed deliveries automatically
Security:
- Verify webhook authenticity using GitHub's HMAC signatures
- Centralized secret management
- Single point of security auditing
- Automatically verify GitHub IP origins (often missed in webhooks implementations)
Observability:
- Track webhook delivery status and latency
- Debug integration issues with detailed logging
- Monitor webhook patterns and volume
Development:
- Test new integrations against real historical events
- Debug webhook handlers without reconfiguring GitHub
- Simulate webhook delivery for development

Architecture

HubProxy consists of three main components:

Webhook Handler: Receives, validates, and forwards GitHub webhooks
Storage Layer: Persists webhook events and delivery status
API Server: Provides REST endpoints for querying and replaying events

The system is designed to be horizontally scalable and can handle high webhook volumes while maintaining strict delivery guarantees.

Development and Testing

Prerequisites

Go 1.24 or later (current codebase uses Go 1.24.2)
SQLite (default), PostgreSQL 14+, or MySQL 8+ for event storage

Database

SQLite is used by default for development, but PostgreSQL or MySQL are recommended for production:

# SQLite (default for development)
hubproxy --db sqlite:.cache/hubproxy.db

# PostgreSQL
hubproxy --db "postgres://user:password@localhost:5432/hubproxy?sslmode=disable"

# MySQL
hubproxy --db "mysql://user:password@tcp(localhost:3306)/hubproxy"

Schema

Here's a simplified version (actual types may vary by database):

CREATE TABLE events (
    id          VARCHAR(255) PRIMARY KEY,    -- GitHub delivery ID
    type        VARCHAR(50) NOT NULL,       -- GitHub event type
    payload     TEXT NOT NULL,              -- Event payload as JSON
    headers     TEXT,                       -- HTTP headers as JSON
    created_at  TIMESTAMP NOT NULL,         -- When the event was received
    forwarded_at TIMESTAMP,                 -- When the event was forwarded
    error       TEXT,                       -- Error message if delivery failed
    repository  VARCHAR(255),               -- Repository full name
    sender      VARCHAR(255),               -- GitHub username
    replayed_from VARCHAR(255),             -- Original event ID if this is a replay
    original_time TIMESTAMP                 -- Original event time if this is a replay
);

-- Indexes for efficient querying
CREATE INDEX idx_created_at ON events (created_at);
CREATE INDEX idx_forwarded_at ON events (forwarded_at);
CREATE INDEX idx_type ON events (type);
CREATE INDEX idx_repository ON events (repository);
CREATE INDEX idx_sender ON events (sender);
CREATE INDEX idx_replayed_from ON events (replayed_from);

Query Options

The storage interface supports filtering events by:

Event type(s)
Repository name
Time range (since/until)
Forwarding status
Sender

Example queries:

// List all events
events, err := storage.ListEvents(QueryOptions{
    Types:      []string{"push", "pull_request"},
    Repository: "owner/repo",
    Since:      time.Now().Add(-24 * time.Hour),
    Forwarded:  true,
})

// List only replayed events
events, err := storage.ListEvents(QueryOptions{
    Forwarded:  false,
})

// List original events that have been replayed
events, err := storage.ListEvents(QueryOptions{
    HasReplayedEvents: true,
})

Querying Replay Events

You can query replayed events using the forwarded filter. For example, to list all replayed events:

events, err := storage.ListEvents(QueryOptions{
    Forwarded:  true,
})

You can also query original events that have been replayed using the HasReplayedEvents filter:

events, err := storage.ListEvents(QueryOptions{
    HasReplayedEvents: true,
})

Event Replay

HubProxy allows you to replay webhook events for testing, recovery, or debugging purposes.

Replay ID Format

Each replayed event has an ID in the format: original-id-replay-uuid

For example:

Original event ID: d2a1f85a-delivery-id-123
Replayed event ID: d2a1f85a-delivery-id-123-replay-abc123

This format ensures:

Easy tracing back to original event
Unique IDs for multiple replays of same event
Clear identification of replayed events

Replay Single Event

// Replay a single event by its ID
event, err := storage.ReplayEvent("d2a1f85a-delivery-id-123")

Development Tools

Development Environment (tools/dev.sh) Sets up a complete development environment with SQLite database and test server.
```
# Start the development environment (required before using other tools)
./tools/dev.sh

# Customize webhook secret
HUBPROXY_WEBHOOK_SECRET=my-secret ./tools/dev.sh

# Customize test server port
./tools/dev.sh --target-port 8083
```
This will:
- Create a SQLite database in .cache/hubproxy.db
- Start a test server to receive forwarded webhooks
- Start the webhook proxy with GitHub IP validation disabled
Default settings:
- Webhook secret: dev-secret (via HUBPROXY_WEBHOOK_SECRET env var)
- Test server port: 8082
- SQLite database: .cache/hubproxy.db

Webhook Simulator (internal/cmd/dev/simulate/main.go) Simulates GitHub webhook events to test the proxy's handling and forwarding.

# Send test webhooks with the default secret
go run internal/cmd/dev/simulate/main.go --secret dev-secret

# Send specific event types
go run internal/cmd/dev/simulate/main.go --secret dev-secret --events push,pull_request

# Add delay between events
go run internal/cmd/dev/simulate/main.go --secret dev-secret --delay 2s

Query Tool (internal/cmd/dev/query/main.go) Inspects and analyzes webhook events stored in the database.

# Show recent events
go run internal/cmd/dev/query/main.go

# Show event statistics
go run internal/cmd/dev/query/main.go --stats

# Filter by event type
go run internal/cmd/dev/query/main.go --type push

# Filter by repository
go run internal/cmd/dev/query/main.go --repo "owner/repo"

Test Server (internal/cmd/dev/testserver/main.go) Simple HTTP server that logs received webhooks for verification. Note: You don't need to run this directly as dev.sh starts it for you.

# Start on default port 8082
go run internal/cmd/dev/testserver/main.go

# Start on custom port
go run internal/cmd/dev/testserver/main.go --port 8083

To verify events are flowing:

# Watch events in real-time
tail -f .cache/testserver.log

Running Tests

# Run all tests
make test

# Run specific package tests
go test ./internal/storage/...
go test ./internal/webhook/...

# Run with race detection
go test -race ./...

Testing Database Connections

# Test PostgreSQL connection
psql "postgres://user:pass@localhost:5432/hubproxy"

# Test MySQL connection
mysql -h localhost -P 3306 -u user -p hubproxy

# Test SQLite database
sqlite3 .cache/hubproxy.db

API Reference

HubProxy provides both REST and GraphQL APIs for querying and replaying webhook events.

API Security

The API server runs on a separate port (default: 8081) from the webhook handler (default: 8080). This separation allows for different security policies:

Webhook Handler (port 8080): Should be publicly accessible to receive GitHub webhooks
API Server (port 8081): Contains sensitive data and control functions, and should be secured

Security Recommendations:

Network Isolation: Keep the API port (8081) behind a firewall or internal network
Reverse Proxy: If exposing the API externally, use a reverse proxy with authentication
Access Control: Consider implementing one of these authentication methods:
- HTTP Basic Authentication via a reverse proxy
- API tokens with a tool like Caddy or Nginx
- VPN or Tailscale for secure network-level access
TLS Encryption: Always use HTTPS for API communications
IP Restrictions: Limit API access to specific IP ranges

Example Nginx Configuration with Basic Auth:

server {
    listen 443 ssl;
    server_name api.hubproxy.example.com;
    
    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;
    
    location / {
        auth_basic "HubProxy API";
        auth_basic_user_file /etc/nginx/.htpasswd;
        proxy_pass ht

Hubproxy

Install / Use

README