Greentea host tests runner (htrun) command line

htrun has extensive command line. In most cases htrun will be run in background:

• driving test binary flashing,
• device reset and
• test execution.

Default binary flashing method is one supported by mbed-enabled devices: binary file is copied on mbed-enabled DUT (Device Under Test) mounted drive (MSD). This procedure will automatically flash device with given binary file content.

Default DUT reset method is one supported by mbed-enabled devices: serial port (CDC) "sendBreak" command resets target MCU on mbed-enabled platform.

Test execution phase will consist of:

• Opening connection between host computer and DUT,
• DUT will send to host preamble with test runner information such as:
  • test environment version,
  • test timeout,
  • preferred host test script (Python script which is used to supervise/instrument test execution),
• Host will spawn host test script and test execution will be instrumented
• Exchange data (in most cases text) between host and DUT,

Command line overview

This chapter will present a few examples of how you can use htrun command line to execute tests. In most cases test automation tools such as Greentea will execute htrun implicitly. There are cases when we want to execute htrun independently. Mostly in situation when we want to:

• debug our code and have binary + host test instrumentation on,
• prototype or
• just want to replace test runner in another OS with one compatible with mbed-enabled devices.

All htrun needs is name of the binary you want to flash and method of flashing!

Useful command line end-to-end examples

Flashing phase operations

Flash binary file /path/to/file/binary.bin using mount point D:. Use serial port COM4 to communicate with DUT:

$ htrun -f /path/to/file/binary.bin -d D: -p COM4

Flash (use shell command copy) binary file /path/to/file/binary.bin using mount point D:. Use serial port COM4 to communicate with DUT:

$ htrun -f /path/to/file/binary.bin -d D: -p COM4 -c copy

Skip flashing phase (e.g. you've already flashed this device with /path/to/file/binary.bin binary). Use serial port COM4 to communicate with DUT:

$ htrun -f /path/to/file/binary.bin -d D: -p COM4 --skip-flashing

DUT-host communication and reset phase

Flash binary file /path/to/file/binary.bin using mount point D:. Use serial port COM4 with baudrate 115200 to communicate with DUT:

$ htrun -f /path/to/file/binary.bin -d D: -p COM4:115200

As above but we will skip reset phase (not so common but in some cases can be used to suppress reset phase for some reasons):

$ htrun -f /path/to/file/binary.bin -d D: -p COM4:115200 --skip-reset

Flash binary file /path/to/file/binary.bin using mount point D:. Use serial port COM4 with default baudrate to communicate with DUT. Do not send __sync key-value protocol synchronization packet to DUT before preamble read:

$ htrun -f /path/to/file/binary.bin -d D: -p COM4 --sync=0

Note: Sync packet management allows you to manipulate the way htrun sends __sync packet(s) to DUT. With current settings we can force on htrun to send __sync packets in this manner:

• --sync=0 - No sync packets will be sent to DUT.
• --sync=-1- __sync packets will be sent unless we will reach timeout or proper response is sent from DUT.
• --sync=N - Where N is integer > 0. Send up to N __sync packets to target platform. Response is sent unless we get response from target platform or timeout occurs.

Global Resource Manager connection

Flash local file /path/to/file/binary.bin to remote device resource (platform K64F) provided by remote_client GRM service available on IP address 10.2.203.31 and port: 8000. Force serial port connection to remote device 9600 with baudrate:

$ htrun -p :9600 -f /path/to/file/binary.bin -m K64F --grm remote_client:10.2.203.31:8000

Command line switch --grm has format: <module_name>:<IP_address>:<port_number>.

• <module_name> - name of Python module to load as remote resource manager.
• <IP_address> and <port_number> - IP address and port of remote resource manager.

Note: Switch -m <platform_name> is required to tell Global Resource Management which platform to request. Note: Command line switch --grm implicitly forces --skip-flashing and --skip-reset because both flags are used for locally available DUTs.

Fast Model connection

This option is designed for htrun to use Arm Fast Models.

The "--fm" option only available when mbed-fastmodel-agent module is installed :

Load local file /path/to/file/binary.elf to onto fastmodel FVP_MPS2_m3 simulators:

$ htrun -f /path/to/file/binary.elf -m FVP_MPS2_M3 --fm DEFAULT

Command line switch format --fm <config_name>.

• <config_name> - either pre-defined CONFIG_NAME from mbedfm or a local config file for the Fast Models.

Note: Switch -m <platform_name> is required to tell this fastmodel connection which Fastmodel to request. Note: Command line switch --fm implicitly forces --skip-flashing and --skip-reset because both flags are used for locally available DUTs.

Miscellaneous

List available host tests names, class names and origin:

$ htrun --list

List available host tests names, class names and origin. Load additional host tests from /path/to/host_tests directory:

$ htrun --list -e /path/to/host_tests

List available reset and flashing plugins:

$ htrun --plugins

Note that some plugin could need some manual installation for extra application before.

Ex: Flash binary file /path/to/file/binary.bin using STM32CubeProgrammer tool. Use serial port COM4 with baudrate 115200 to communicate with DUT:

htrun -c stprog -f /path/to/file/binary.bin -p COM4:115200

In this example, STM32_Programmer_CLI application has to be in the environment path.

Installation

htrun is distributed along with greentea-host module as a dependency sources and its command line tool called htrun.

Installation from PyPI (Python Package Index)

greentea-host module is distributed via PyPI. We recommend you use the application pip.

Note: Python 2.7.9 onwards include pip by default, so you may have pip already.

To install mbed-ls from PyPI use command:

$ pip install greentea-host --upgrade

Installation from Python sources

To install the mbed test suite, first clone the greentea repository:

$ git clone https://github.com/ARMmbed/greentea.git

Change the directory to the greentea directory:

$ cd greentea

Now you are ready to install htrun:

$ python setup.py install

Note: setup.py will install both greentea (gt) and htrun command line tools.

Checking installation

To check whether the installation was successful try running the htrun --help command and check that it returns information (you may need to restart your terminal first):

$ htrun --help
Usage: htrun [options]

Flash, reset and perform host supervised tests on Mbed enabled platforms

Options:
  -h, --help            show this help message and exit

Greentea host tests runner (htrun)

Mbed's test suite (codenamed Greentea) supports the test supervisor concept. This concept is realized by this module. htrun is a collection of host tests. Host test is script written in Python, which is executed in parallel with the test suite runner (a binary running on the target hardware / device under test) to monitor the test execution's progress or to control the test flow (interaction with the mbed device under test - DUT). The host test is also responsible for grabbing the test result, or deducing it from the test runner's behavior.

Key-value protocol was developed and is used to provide communication layer between DUT (device under test) and host computer. Key-value protocol defined host computer as master and DUT as slave.

• Slave side APIs and key-value protocol implementation is encapsulated in greentea-client module.
• Master side APIs and key-value protocol is encapsulated in htrun.

htrun responsibilities are:

• Flash mbed device with given binary.
• Reset mbed device after flashing to start test suite execution.
• Use key-value protocol to handshake with device and make sure correct host test script is executed to supervise test suite execution.
• Run key-value protocol state machine and execute event callbacks.
• Monitor serial port traffic to parse valid key-value protocol events.
• Make decision if test test suite passed / failed / returned error.
• Provide command line tool interface, command: htrun after module installation (on host).
• Provide few basic host test implementations which can be used out of the box for test development. For example the basic host test (called default or default_auto) just parses events from DUT and finished host test execution when end event is received. Other included in this module host tests can help you to test timers or RTC.

Key-value protocol overview

• Text based protocol, format {{KEY;VALUE}}}.
• Master-slave mode where host is master and DUT is slave.

Design draft

• Simple key-value protocol is introduced. It is used to communicate between DUT and host. Protocol main features:
• Protocol introduced is