DFrame

Distributed load-testing Framework for .NET and Unity.

This library allows you to write distributed load test scenarios in plain C#, no needs weird gui, dsl, xml, json, yaml. In addition to HTTP/1, you can test HTTP/2, gRPC, MagicOnion, Photon, or original network transport by writing in C#.

DFrame is similar as Locust, combination of two parts, DFrame.Controller(built by Blazor Server) as Web UI and DFrame.Worker as C# test scenario script. DFrame is providing as a library however you can bootup easily if you are faimiliar with C#.

// Install-Package DFrame
using DFrame;

DFrameApp.Run(7312, 7313); // WebUI:7312, WorkerListen:7313

public class SampleWorkload : Workload
{
    public override async Task ExecuteAsync(WorkloadContext context)
    {
        Console.WriteLine($"Hello {context.WorkloadId}");
    }
}

You can now open your browser and run the tests you have set up. It can be used as a single execution tool like Ab, but the distribution mechanism is very simple. When you start the Worker application, it will go to the connect address of the Controller by MagicOnion(grpc-dotnet). That's it, the connection is complete. Now all you have to do is wait for the command from the web UI.

DFrame.Worker also supports Unity. This means that by deploying it on a large number of Headless Unity or device farms, we can load test even network frameworks that only work with Unity.

Don't forget about performance. It is very important to hit a lot of RPS on single machine. Many of the major load testing tools are not very powerful(except for wrk). DFrame is highly optimized and also brings out the full power of compiled C# code.

<!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

Table of Contents

• Getting started
• Controller and Worker
  • DFrame.Controller
  • DFrame.Worker
• Workload
  • WorkloadContext
• Mode
  • Request
  • Repeat
  • Duration
  • Infinite
• Options
  • DFrameControllerOptions
  • DFrameWorkerOptions
    • Metadata
• DFrameApp/DFrameAppBuilder
• Persistent execute results
• REST API for Automation
• Unity
• Controller event handling
• License

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

Getting started

For .NET, use NuGet. For Unity, please read Unity section.

Install-Package DFrame

DFrameApp.Run is most simple entry point of DFrame. It runs DFrame.Controler and DFrame.Worker in single binary.

DFrame calls a test scenario a Workload. Your test scenario implements Workload and Task ExecuteAsync(WorkloadContext context).

DFrame.Controler needs Microsoft.NET.Sdk.Web so you can start from ASP.NET Core Empty Template and add <RequiresAspNetWebAssets>true</RequiresAspNetWebAssets> to PropertyGroup. or create from Console template and change csproj like this.

<!-- Use Sdk.Web -->
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <!-- Add this line for .NET 10 -->
    <RequiresAspNetWebAssets>true</RequiresAspNetWebAssets>
  </PropertyGroup>
</Project>

using DFrame;

DFrameApp.Run(7312, 7313); // WebUI:7312, WorkerListen:7313

public class SampleWorkload : Workload
{
    public override async Task ExecuteAsync(WorkloadContext context)
    {
        Console.WriteLine($"Hello {context.WorkloadId}");
    }
}

Open the browser http://localhost:7312, Workload select-box has this SampleWorkload.

ExecuteAsync is invoked "Total Request" times. Concurrency is sometimes referred to as Virtual User in other frameworks. In DFrame, create N workloads on single worker and invoke ExecuteAsync in parallel.

Other overloads, Workload has SetupAsync, TeardownAsync and Complete. For example, simple gRPC test is here.

public class GrpcTest : Workload
{
    GrpcChannel? channel;
    Greeter.GreeterClient? client;

    public override async Task SetupAsync(WorkloadContext context)
    {
        channel = GrpcChannel.ForAddress("http://localhost:5027");
        client = new Greeter.GreeterClient(channel);
    }

    public override async Task ExecuteAsync(WorkloadContext context)
    {
        await client!.SayHelloAsync(new HelloRequest(), cancellationToken: context.CancellationToken);
    }

    public override async Task TeardownAsync(WorkloadContext context)
    {
        if (channel != null)
        {
            await channel.ShutdownAsync();
            channel.Dispose();
        }
    }
}

You can also accept parameters, so you can create something like passing an arbitrary URL. In the constructor, you can accept parameters or an instance injected by DI.

using DFrame;
using Microsoft.Extensions.DependencyInjection;

// use builder can configure services, logging, configuration, etc.
var builder = DFrameApp.CreateBuilder(7312, 7313);
builder.ConfigureServices(services =>
{
    services.AddSingleton<HttpClient>();
});
await builder.RunAsync();

public class HttpGetString : Workload
{
    readonly HttpClient httpClient;
    readonly string url;

    // HttpClient is from DI, URL is passed from Web UI
    public HttpGetString(HttpClient httpClient, string url)
    {
        this.httpClient = httpClient;
        this.url = url;
    }

    public override async Task ExecuteAsync(WorkloadContext context)
    {
        await httpClient.GetStringAsync(url, context.CancellationToken);
    }
}

If you want to test a simple HTTP GET/POST/PUT/DELETE, you can enable IncludeDefaultHttpWorkload, which will add a workload that accepts url and body parameters.

using DFrame;

var builder = DFrameApp.CreateBuilder(7312, 7313);
builder.ConfigureWorker(x =>
{
    x.IncludesDefaultHttpWorkload = true;
});
builder.Run();

This option is useful if you want to try out a DFrame.

Controller and Worker

Worker connections means multiple processes. If they are running on different servers, they can be executed concurrently from distributed servers. The Controller must always be a single process, but the Worker can launch multiple processes.

There are two ways to separate the Controller from the Worker. The first is to simply separate the projects.

The other ways is to switch modes with command line arguments. I recommend this one as it makes local development easier. DFrameApp.CreateBuilder has RunAsync(run both), RunControllerAsync(run only controller), RunWorkerAsync(run only worker).

using DFrame;

var builder = DFrameApp.CreateBuilder(5555, 5556); // portWeb, portListenWorker

if (args.Length == 0)
{
    // local, run both(host WebUI on http://localhost:portWeb)
    await builder.RunAsync();
}
else if (args[0] == "controller")
{
    // listen http://*:portWeb as WebUI and http://*:portListenWorker as Worker listen gRPC
    await builder.RunControllerAsync();
}
else if (args[0] == "worker")
{
    // worker connect to (controller) address.
    // You can also configure from appsettings.json via builder.ConfigureWorker((ctx, options) => { options.ControllerAddress = "" });
    await builder.RunWorkerAsync("http://foobar:5556");
}

DFrame.Controller

For minimizes dependency, you can only reference DFrame.Controller instead of DFrame.

Install-Package DFrame.Controller

Controller project must use Microsoft.NET.Sdk.Web and add <RequiresAspNetWebAssets>true</RequiresAspNetWebAssets> to PropertyGroup.

If you want to use DFrame.Controller instead of DFrameApp, build it from WebApplicationBuilder and RunDFrameControllerAsync().

using DFrame;
using Microsoft.AspNetCore.Builder;

var builder = WebApplication.CreateBuilder(args);
await builder.RunDFrameControllerAsync();

DFrame.Controller open two addresses, Http/1 is for Web UI(built on Blazor Server), Http/2 is for worker clusters(built on MagicOnion(gRPC)). You have to add appsettings.json(and CopyToOutputDirectory) to configure address.

{
  "Kestrel": {
    "Endpoints": {
      "Http": {
        "Url": "http://localhost:7312",
        "Protocols": "Http1"
      },
      "Grpc": {
        "Url": "http://localhost:7313",
        "Protocols": "Http2"
      }
    }
  }
}

DFrameApp/DFrameAppBuilder.Run() has string? controllerAddress = null parameter. If does not pass any value, DFrame.Worker connect to http://localhost:portListenWorker. If you want to connect other server, must pass controllerAddress.

// controller listen worker on http://*:7313 and worker connect to "http://999.99