Medusa

Medusa is a tool to easily parallelize execution of Robot Framework test suites.

If you have a lot of Robot Framework tests that take a non-negligible amount of time, you can benefit greatly from running them in parallel. Medusa uses suite metadata to start suites in parallel dynamically while preventing resource usage conflicts. Suites can be assigned to sequentially executed stages and can be run multiple times with different variables, even in parallel.

This could be used to execute one or multiple suites against multiple target devices or endpoints simultaneously, or even to execute a single suite sequentially multiple times with different configuration.

Parallelization happens on suite level, it is not (currently) possible to run tests within a single suite in parallel. It is also not (currently) possible to have one suite directly depend on another suite to finish first, though you could use separate stages to achieve this. Consider using pabot if you need these features.

Medusa is developed by INSYS icom GmbH and licensed as Open Source Software, see the License section below for details. INSYS icom does not provide support for Medusa, see Reporting Bugs if you found a problem.

Table of Contents

1. Installation
   1. User
   2. Developer
2. Quick Start
3. Command line usage
4. Suite metadata
5. Reporting bugs
6. Contributing features
7. License

Installation

NOTE: Medusa is currently only tested on Linux and does not work on Windows 11, see #4.

Make sure you have installed python3 pip and venv.

User

python3 -m venv .venv  # Create a venv in .venv (if not done already)
. .venv/bin/activate   # Enter the venv

# Install medusa from PyPI.org
python3 -m pip install robotframework-medusa
# OR install it from source
python3 -m pip install .

# use medusa...

deactivate                # Exit the venv (when you are done)

Developer

python3 -m venv .venv  # Create a venv in .venv (if not done already)
. .venv/bin/activate   # Enter the venv
python3 -m pip install -e '.[dev]'  # Install editable with dev dependencies

# use medusa, do your developing...

make format            # Run formatter
make check             # Run type checker and linter
make fix               # Auto-fix linter suggestions (if possible)
make test              # Run tests
deactivate             # Exit the venv (when you are done)

Quick Start

Add at least the required metadata medusa:stage and medusa:deps to your suite(s). Optionally add medusa:timeout for suite-specific timeouts or medusa:for for multiplying suites with different variables.

*** Settings ***
Documentation    Stage 1, multiply suite with three different dependencies and
...              input values, timeout 5min (soft), 30s (hard), 5s (kill)
...              First suite:  $DEP=foo    $INPUT=input1
...              Second suite: $DEP=bar    $INPUT=input2
...              Third suite:  $DEP=baz    $INPUT=input3
Metadata    medusa:stage      1
Metadata    medusa:deps       ${DEP}
Metadata    medusa:for        ${DEP}    ${INPUT}    IN    &{DEP_INPUT_DICT}
Metadata    medusa:timeout    300,30,5

*** Variables ***
${DEP}               ${None}  # Set by medusa:for
${INPUT}             ${None}  # Set by medusa:for
&{DEP_INPUT_DICT}    foo=input1    bar=input2    baz=input3

Usage summary:

• medusa stats: View information about your suite(s)
• medusa run: Run the suite(s)
• Use robot options with -- as a separator after medusa options: medusa run -- -i security my_suite.robot

Results are stored in the results/ directory in a date/timestamped subdirectory by default, this can be changed with the -d option. See medusa -h and the detailed documentation below for more information.

Command line usage

Run medusa --help for full usage information, this section is just a rough overview. Medusa supports two main actions, stats and run. You can use help to get additional info about some options and version to output medusa's version.

The stats and run commands accept arguments just like robot. Example with just suites as arguments:

medusa stats single_suite.robot my_suite_dir/

You can also use (almost) all options that robot accepts. You need to write -- before any robot options in order to separate them from Medusa's own options. In this example, we use robot's --dryrun option:

#                     Separator ──┲┓
medusa run --outputdir customdir/ -- --dryrun my_suite_dir/
#          ┗━━ Medusa Option ━━━┛    ┗━━━ Robot Option ━━━┛

medusa stats ...

Medusa reads the specified suite(s) and outputs information about them. By default, only a short summary of stats is given but more information can be shown with the -s or --select option. This includes infomation about recognised stages, dependencies and suites, as well as tags. Additional options:

• Filter suites by stage/dependency with -f or --filter
• Output additional information with -s or --select

medusa run ...

Medusa reads the specified suite(s) and executes them. Stages are sorted alphabetically and executed sequentially. Within each stage, medusa dynamically starts as many suite in parallel as possible without starting suites with overlapping dependencies. Additional options:

• Change the output directory with -d or --outputdir
• Filter suites by stage/dependency with -f or --filter
• Set global soft/hard/kill timeouts with -t or --timeout (can be overriden with suite metadata)

Suite metadata

The order and parallelisation of suites is determined entirely by suite metadata. For this reason, every suite needs to have at least medusa:stage and medusa:deps metadata configured. The medusa:for and medusa:timeout metadata is optional.

The below examples use the $VAR escaped variable syntax but the regular ${VAR} syntax works too.

medusa:stage (required)

Each suite needs to be assigned to a stage using the medusa:stage metadata key. Stages are executed sequentially in alphanumeric order, meaning that two suites in different stages will never run in parallel. After all suites in one stage finished executing, the next stage is executed.

Example:

*** Settings ***
Metadata    medusa:stage    1_Example

Given these three suites:

• Suite1 with medusa:stage = 1_Example
• Suite2 with medusa:stage = 1_Example
• Suite3 with medusa:stage = 2_Potato

The suites Suite1 and Suite2 will be executed first, possibly in parallel (if their dependencies allow it). Once both of them finished, Suite3 will be executed.

medusa:deps (required)

Each suite needs to declare dependencies using the medusa:deps metadata key. If two suites in the same stage have overlapping dependencies, they are not executed in parallel.

Example:

*** Settings ***
Metadata    medusa:deps    foo    ${BAR}    @{BAZ}

*** Variables ***
${BAR}    bar
@{BAZ}    baz    buzz    butz

As you can see, medusa:deps takes a list of values separated by two or more spaces. You can either directly write the value or use scalar or list variables. In the case of list variables, the list is simply flattened. This means that the above example is equivalent to:

*** Settings ***
Metadata    medusa:deps    foo    bar    baz    buzz    butz

Given these three suites in the same stage:

• Suite1 with medusa:deps = foo, bar
• Suite2 with medusa:deps = bar, baz
• Suite3 with medusa:deps = buzz, butz

The suites Suite1 and Suite2 will not be executed in parallel because they both have the dependency bar. Suite3 will be executed in parallel to the other two because it does not have any dependencies in common with them.

Dynamic dependencies

Medusa can also pick a dependency from a list of options at runtime, depending on whether one of the options is available. This can be done with the ANY $ITEM IN $ITERABLE syntax. $ITERABLE has to be any iterable type containing str values, such as a list or set. Here is an example for a suite that can run on any one device out of a list:

*** Settings ***
Metadata    medusa:deps    ANY $DUT IN $DEVICES

*** Variables
@{DEVICES}    dut1    dut2    dut3  # List of options
${DUT}    ${None}                   # Determined at runtime by medusa

When this suite is executed with medusa, ${DUT} could have any one of the three values dut1, dut2 or dut3 depending on which ones is available (not currently used by another suite).

Dynamic dependencies can also be combined with medusa:for, for example in order to run the same suite on two devices from different lists of options.

medusa:for (optional)

The medusa:for metadata key can be used to execute one suite multiple times with differently set suite variables. This could be used to run the same test suite in parallel against multiple endpoints or test devices or with slightly differing configuration. medusa:for expects the format $TARGET [$TARGET...] IN $SOURCE. The $SOURCE must be either a Mapping (such as dict) or an Iterable (such as list or set) of input values and the $TARGETs are variable names to assign the input values to.

Example with one target variable and simple input list:

*** Settings ***
Metadata    medusa:for    $DUT    IN    $DUTS

*** Variables ***
@{DUTS}    foo    bar  # Source (input values)
${DUT}    ${None}      # Target variable

The above example suite will be executed twice, once with ${DUT} = foo and once with ${DUT} = bar.

Example with three target variables and two-dimensional input list