RSSHub

A high-performance CLI RSS feed aggregator built with Go, featuring concurrent feed processing via configurable worker pools, PostgreSQL storage, and runtime configuration updates.

Features

• Concurrent Feed Processing - Efficient worker pool architecture for parallel RSS feed fetching
• Dynamic Configuration - Change fetch intervals and worker count without restarting
• PostgreSQL Storage - Reliable persistence with automatic migrations
• Clean Architecture - Well-structured codebase following domain-driven design principles
• Docker Support - Easy deployment with Docker Compose
• Graceful Shutdown - Proper cleanup and resource management

Requirements

• Go 1.21+
• PostgreSQL 15+
• Docker & Docker Compose (optional)

Installation

Using Docker Compose (Recommended)

git clone <repository-url>
cd rsshub
cp .env.example .env
docker-compose up -d

Local Installation

git clone <repository-url>
cd rsshub
go build -o rsshub ./cmd/rsshub

Quick Start

1. Start PostgreSQL (if running locally)

docker run -d \
  --name rsshub-postgres \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=changeme \
  -e POSTGRES_DB=rsshub \
  -p 5432:5432 \
  postgres:15-alpine

2. Set Environment Variables

export POSTGRES_HOST=localhost
export POSTGRES_PORT=5432
export POSTGRES_USER=postgres
export POSTGRES_PASSWORD=changeme
export POSTGRES_DBNAME=rsshub

3. Build and Run

go build -o rsshub ./cmd/rsshub
./rsshub --help

Usage

Add RSS Feeds

./rsshub add --name "tech-crunch" --url "https://techcrunch.com/feed/"
./rsshub add --name "hacker-news" --url "https://news.ycombinator.com/rss"

Start Background Fetching

In terminal 1:

./rsshub fetch
# Output: The background process for fetching feeds has started (interval = 3 minutes, workers = 3)

Manage Configuration (while fetcher is running)

In terminal 2:

# Change fetch interval
./rsshub set-interval 2m

# Adjust worker pool size
./rsshub set-workers 5

View Feeds and Articles

# List all feeds
./rsshub list

# List 5 most recent feeds
./rsshub list --num 5

# Show latest articles from a feed
./rsshub articles --feed-name "tech-crunch" --num 10

# Delete a feed
./rsshub delete --name "tech-crunch"

Stop the Aggregator

Press Ctrl+C in the terminal running rsshub fetch for graceful shutdown.

CLI Commands

| Command | Description | Example | |---------|-------------|---------| | fetch | Start background RSS fetching | ./rsshub fetch | | add | Add new RSS feed | ./rsshub add --name "feed" --url "https://..." | | set-interval | Change fetch interval | ./rsshub set-interval 5m | | set-workers | Resize worker pool | ./rsshub set-workers 10 | | list | List RSS feeds | ./rsshub list --num 5 | | delete | Delete RSS feed | ./rsshub delete --name "feed" | | articles | Show latest articles | ./rsshub articles --feed-name "feed" --num 5 | | --help | Display help | ./rsshub --help |

Configuration

Environment variables (see .env.example):

| Variable | Description | Default | |----------|-------------|---------| | CLI_APP_TIMER_INTERVAL | Feed fetch interval | 3m | | CLI_APP_WORKERS_COUNT | Number of concurrent workers | 3 | | POSTGRES_HOST | PostgreSQL host | localhost | | POSTGRES_PORT | PostgreSQL port | 5432 | | POSTGRES_USER | Database user | postgres | | POSTGRES_PASSWORD | Database password | changeme | | POSTGRES_DBNAME | Database name | rsshub |

Architecture

The project follows Clean Architecture principles:

rsshub/
├── cmd/
│   └── rsshub/         # Application entry point
├── internal/
│   ├── domain/         # Business entities and interfaces
│   ├── app/            # Application services and use cases
│   ├── adapter/        # External adapters (DB, RSS parser)
│   └── cli/            # CLI command handlers
├── migrations/         # Database migrations
├── docker-compose.yml  # Container orchestration
└── .env.example       # Environment configuration template

Example RSS Feeds

• TechCrunch: https://techcrunch.com/feed/
• Hacker News: https://news.ycombinator.com/rss
• BBC World: http://feeds.bbci.co.uk/news/world/rss.xml
• Ars Technica: http://feeds.arstechnica.com/arstechnica/index
• The Verge: https://www.theverge.com/rss/index.xml

Development

Run with Race Detection

go run -race ./cmd/rsshub

Format Code

gofumpt -w .

Build

go build -o rsshub ./cmd/rsshub