Scenarigo

<a href="https://github.com/scenarigo/scenarigo"> <p align="center"> <img alt="Scenarigo" src="https://user-images.githubusercontent.com/2238852/205980597-37eaaf03-fd35-4a04-93c4-884c95f48df3.png" width="485px"> </p> </a>

A scenario-based API testing tool for HTTP/gRPC server.

Overview

Scenarigo is a scenario-based API testing tool for HTTP/gRPC server. It is written in Go and provides a plugin feature that enables you to extend by writing Go code. You can write test scenarios as YAML files and executes them.

title: get scenarigo repository
vars:
  user: scenarigo
  repo: scenarigo
steps:
- title: get repository
  protocol: http
  request:
    method: GET
    url: 'https://api.github.com/repos/{{vars.user}}/{{vars.repo}}'
  expect:
    code: OK
    body:
      id: '{{int($) > 0}}'
      name: '{{vars.repo}}'

Features

• Multi-Protocol Support - Test both HTTP/REST and gRPC APIs
• YAML-based Scenarios - Write test scenarios in a readable, declarative format
• Template Strings - Dynamic value generation and validation with template expressions
• Plugin System - Extend functionality by writing custom Go plugins
• Variables and Secrets - Manage test data with variable scoping and secret masking
• Retry Policies - Built-in retry with constant or exponential backoff strategies
• Conditional Execution - Control test flow with conditional step execution
• ytt Integration - Advanced templating and overlay capabilities for test scenarios

Quick Start

Installation

Install Scenarigo using Go:

$ go install github.com/scenarigo/scenarigo/cmd/scenarigo@latest

Your First Test

Create a simple test scenario file hello.yaml:

title: Hello Scenarigo
steps:
- title: Check GitHub API
  protocol: http
  request:
    method: GET
    url: https://api.github.com/repos/scenarigo/scenarigo
  expect:
    code: OK
    body:
      name: scenarigo

Running Tests

Create a configuration file and run the test:

# Initialize configuration
$ scenarigo config init

# Run the test
$ scenarigo run hello.yaml
ok      hello.yaml     0.123s

That's it! You've just run your first Scenarigo test. Continue reading to learn more advanced features.

Installation (Detailed)

go install command (recommend)

$ go install github.com/scenarigo/scenarigo/cmd/scenarigo@latest

from release page

Go to the releases page and download the zip file. Unpack the zip file, and put the binary to a directory in your $PATH.

You can download the latest command into the ./scenarigo directory with the following one-liner code. Place the binary ./scenarigo/scenarigo into your $PATH.

$ version=$(curl -s https://api.github.com/repos/scenarigo/scenarigo/releases/latest | jq -r '.tag_name') && \
    go_version=$(echo -n $(curl -s 'https://go.dev/VERSION?m=text' | head -n 1)) && \
    curl -sLJ https://github.com/scenarigo/scenarigo/releases/download/${version}/scenarigo_${version}_${go_version}_$(uname)_$(uname -m).tar.gz -o scenarigo.tar.gz && \
    mkdir ./scenarigo && tar -zxvf ./scenarigo.tar.gz -C ./scenarigo && rm scenarigo.tar.gz

Notes: If you use the plugin mechanism, the scenarigo command and plugins must be built using the same version of Go.

Setup

You can generate a configuration file scenarigo.yaml via the following command.

$ scenarigo config init

schemaVersion: config/v1

# global variables
vars:
  endpoint: http://api.example.com

scenarios: [] # Specify test scenario files and directories.

pluginDirectory: ./gen    # Specify the root directory of plugins.
plugins:                  # Specify configurations to build plugins.
  plugin.so:              # Map keys specify plugin output file path from the root directory of plugins.
    src: ./path/to/plugin # Specify the source file, directory, or "go gettable" module path of the plugin.

output:
  verbose: false # Enable verbose output.
  colored: false # Enable colored output with ANSI color escape codes. It is enabled by default but disabled when a NO_COLOR environment variable is set (regardless of its value).
  summary: false # Enable summary output.
  report:
    json:
      filename: ./report.json # Specify a filename for test report output in JSON.
    junit:
      filename: ./junit.xml   # Specify a filename for test report output in JUnit XML format.

Usage

scenarigo run executes test scenarios based on the configuration file.

schemaVersion: config/v1

scenarios:
- github.yaml

title: get scenarigo repository
steps:
- title: GET https://api.github.com/repos/scenarigo/scenarigo
  vars:
    user: scenarigo
    repo: scenarigo
  protocol: http
  request:
    method: GET
    url: "https://api.github.com/repos/{{vars.user}}/{{vars.repo}}"
  expect:
    code: OK
    body:
      name: "{{vars.repo}}"

$ scenarigo run
ok      github.yaml     0.068s

Alternatively, provide the paths to specific test files as arguments.

$ scenarigo run github.yaml

You can see all commands and options by scenarigo help.

scenarigo is a scenario-based API testing tool.

Usage:
  scenarigo [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  config      manage the scenarigo configuration file
  dump        dump test scenario files
  help        Help about any command
  list        list the test scenario files
  plugin      provide operations for plugins
  run         run test scenarios
  version     print scenarigo version

Flags:
  -c, --config string   specify configuration file path (read configuration from stdin if specified "-")
  -h, --help            help for scenarigo
      --root string     specify root directory (default value is the directory of configuration file)

Use "scenarigo [command] --help" for more information about a command.

How to write test scenarios

You can write test scenarios easily in YAML. A test scenario consists of steps that are executed sequentially from top to bottom. Each step represents an API request (HTTP or gRPC) and its expected response.

Scenarigo supports testing both HTTP/REST and gRPC APIs. The following sections describe how to write tests for each protocol.

HTTP Testing

Send HTTP requests

This simple example has a step that sends a GET request to http://example.com/message.

title: check /message
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message

To send a query parameter, add it directly to the URL or use the query field.

title: check /message
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message
    query:
      id: 1

You can use other methods to send data to your APIs.

title: check /message
steps:
- title: POST /message
  protocol: http
  request:
    method: POST
    url: http://example.com/message
    body:
      message: hello

By default, Scenarigo will send body data as JSON. If you want to use other formats, set the Content-Type header.

title: check /message
steps:
- title: POST /message
  protocol: http
  request:
    method: POST
    url: http://example.com/message
    header:
      Content-Type: application/x-www-form-urlencoded
    body:
      message: hello

Available Content-Type header to encode request body is the following.

• application/json (default)
• text/plain
• application/x-www-form-urlencoded

Check HTTP responses

You can test your APIs by checking responses. If the result differs from the expected values, Scenarigo aborts the execution of the test scenario and notifies the error.

Scenarigo provides three ways to validate response values in the expect field:

1. Exact Matching - Compare values directly for equality
2. Template Expressions - Use conditional expressions with the actual value $
3. Assertion Functions - Use built-in assertion functions for common validations

Exact Matching

The simplest way to validate responses is to specify the expected values directly. Scenarigo will compare them for exact equality.

title: exact matching
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message
    query:
      id: 1
  expect:
    code: OK
    header:
      Content-Type: application/json; charset=utf-8
    body:
      id: 1
      message: hello

This method is best when you know the exact expected value and want a simple equality check.

Template Expressions

For more flexible validations, you can use template string expressions with the actual value represented by $. This allows you to write conditional expressions and perform calculations.

title: template expressions
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message
    query:
      id: 1
  expect:
    code: OK
    header:
      Content-Type: application/json; charset=utf-8
    body:
      id: '{{int($) > 0}}'                                    #