Valir

High-performance background job processing for .NET. Redis-backed queue with atomic operations, graceful shutdown, distributed locks, rate limiting, OpenTelemetry tracing, and transactional outbox pattern. Supports Kafka, RabbitMQ, and Azure Service Bus.

Generate Convert Improve

Install / Use

/learn @Taiizor/Valir

About this skill

Quality Score

0/100

README

<img src="Valir.png" alt="Valir Logo" width="128" height="128"> <h1 align="center">Valir</h1> A modular .NET distributed job queue framework with Redis backend, supporting at-least-once delivery, priority queues, transactional outbox, and event-driven architectures via Kafka/RabbitMQ/Azure Service Bus. <a href="https://github.com/Taiizor/Valir/actions/workflows/ci.yml"> <img src="https://github.com/Taiizor/Valir/actions/workflows/ci.yml/badge.svg" alt="Build"> </a> <a href="https://www.nuget.org/packages/Valir.Redis"> <img src="https://img.shields.io/nuget/v/Valir.Redis?color=blue&label=NuGet" alt="NuGet"> </a> <a href="https://www.nuget.org/packages/Valir.Redis"> <img src="https://img.shields.io/nuget/dt/Valir.Redis?color=green&label=Downloads" alt="Downloads"> </a> <a href="https://dotnet.microsoft.com"> <img src="https://img.shields.io/badge/.NET-10.0-512BD4" alt=".NET"> </a> <a href="LICENSE"> <img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License"> </a> <a href="#features">Features</a> • <a href="#quick-start">Quick Start</a> • <a href="#packages">Packages</a> • <a href="#documentation">Documentation</a> • <a href="#contributing">Contributing</a>

Features

| Feature | Description | |---------|-------------| | ✅ At-Least-Once Delivery | Idempotency keys for reliable processing | | ✅ Priority Queues | Higher priority jobs processed first | | ✅ Batch Operations | Enqueue thousands of jobs efficiently | | ✅ Graceful Shutdown | Drain mode for zero job loss | | ✅ Transactional Outbox | Atomic job creation with your DB | | ✅ Distributed Locks | Redis-backed coordination | | ✅ Rate Limiting | Sliding window algorithm | | ✅ OpenTelemetry | Native tracing support | | ✅ TUI Dashboard | Interactive worker console |

Quick Start

Installation

# Core packages
dotnet add package Valir.Redis
dotnet add package Valir.AspNet

# Event Bus (choose one)
dotnet add package Valir.Brokers.Kafka
dotnet add package Valir.Brokers.RabbitMQ
dotnet add package Valir.Brokers.AzureSB

Producer (Web API)

// Program.cs
builder.Services.AddValir(options =>
{
    options.RedisConnectionString = "localhost:6379";
});

// Enqueue a job
app.MapPost("/send-email", async (IJobQueue queue, EmailRequest req) =>
{
    byte[] payload = JsonSerializer.SerializeToUtf8Bytes(req);
    string jobId = await queue.EnqueueAsync(
        type: "send-email",
        payload: payload,
        priority: 5,
        idempotencyKey: $"email-{req.UserId}"
    );
    return Results.Ok(new { JobId = jobId });
});

Consumer (Worker)

// Define your job handler
public class EmailJobHandler : IJobHandler<EmailRequest>
{
    public async Task HandleAsync(EmailRequest request, JobContext context)
    {
        // Process the job
        await SendEmailAsync(request, context.CancellationToken);
        
        Console.WriteLine($"Email sent to {request.Email}");
    }
}

// Program.cs - Register and run
builder.Services.AddValir(options =>
{
    options.RedisConnectionString = "localhost:6379";
    options.Concurrency = 4;
});

builder.Services.AddSingleton<IJobHandler<EmailRequest>, EmailJobHandler>();

// Start worker runtime
var worker = app.Services.GetRequiredService<WorkerRuntime>();
await worker.StartAsync(CancellationToken.None);

Or use the sample worker with TUI:

dotnet run --project samples/Valir.Sample.Worker -- --redis localhost:6379 --concurrency 4

Packages

| Package | Description | NuGet | |---------|-------------|-------| | Valir.Abstractions | Core interfaces | | | Valir.Core | Worker runtime, retry policies | | | Valir.Redis | Redis job queue, distributed lock | | | Valir.AspNet | ASP.NET Core integration | | | Valir.EntityFrameworkCore | Transactional Outbox pattern | | | Valir.Brokers.Kafka | Apache Kafka adapter | | | Valir.Brokers.RabbitMQ | RabbitMQ adapter | | | Valir.Brokers.AzureSB | Azure Service Bus adapter | |

Architecture

graph LR
    subgraph Producer
        A[Web API]
    end
    
    subgraph Storage
        B[(Redis<br/>Job Queue)]
    end
    
    subgraph Consumer
        C[Worker]
        D[Handler<br/>Your Code]
    end
    
    subgraph Events
        E[Event Bus<br/>Kafka/RMQ/Azure]
    end
    
    A -->|Enqueue| B
    B -->|Claim| C
    C --> D
    A -->|Publish| E

Configuration

services.AddValir(options =>
{
    // Redis connection
    options.RedisConnectionString = "localhost:6379";
    options.KeyPrefix = "valir:";
    
    // Worker settings
    options.Concurrency = 4;
    options.DefaultMaxAttempts = 3;
    options.RetryBaseDelay = TimeSpan.FromSeconds(10);
    
    // Timeouts
    options.DefaultVisibilityTimeout = TimeSpan.FromSeconds(30);
    options.ShutdownTimeout = TimeSpan.FromSeconds(30);
});

Event Bus Adapters

<details> <summary>Kafka</summary>

services.AddValirKafka(options =>
{
    options.BootstrapServers = "localhost:9092";
    options.GroupId = "my-service";
    options.EnableAutoCommit = false; // At-least-once
});

</details> <details> <summary>RabbitMQ</summary>

services.AddValirRabbitMQ(options =>
{
    options.HostName = "localhost";
    options.UserName = "guest";
    options.Password = "guest";
    options.ExchangeName = "valir.events";
});

</details> <details> <summary>Azure Service Bus</summary>

services.AddValirAzureServiceBus(options =>
{
    options.ConnectionString = "Endpoint=sb://...";
    options.MaxConcurrentCalls = 10;
});

</details>

Transactional Outbox

Ensure job creation is atomic with your database transaction:

// Register outbox
services.AddValirOutbox<AppDbContext>();

// In your service
public async Task CreateOrderAsync(Order order)
{
    _context.Orders.Add(order);
    
    // Job is written to outbox table (same transaction)
    await _outboxQueue.EnqueueAsync("process-order", payload);
    
    await _context.SaveChangesAsync(); // Atomic!
}
// Background processor pushes to Redis automatically