ArduinoUnit

ArduinoUnit is a testing framework for Arduino projects. It supports Arduino, ESP8266 and ESP32 as well as "en vitro" development system (vs embedded target) testing.

Getting Started

Install the library from the Arduino IDE. From the menu, navigate:

• Sketch->Include Library->Manage Libraries...
• Search for "arduinounit"
• Install

After this, examples should be available from File->Examples in the "Examples from Custom Libraries" section.

Here is a simple unit testing sketch:

#line 2 "sketch.ino"
#include <ArduinoUnit.h>

test(ok) 
{
  int x=3;
  int y=3;
  assertEqual(x,y);
}

test(bad)
{
  int x=3;
  int y=3;
  assertNotEqual(x,y);
}

void setup()
{
  Serial.begin(9600);
  while(!Serial) {} // Portability for Leonardo/Micro
}

void loop()
{
  Test::run();
}

Upload this sketch to the Arduino (using the 'Upload' button, File | Upload or Ctrl+U).

Turn on the Serial Monitor (using the 'Serial Monitor' button, Tools | Serial Monitor or Ctrl+Shift+M) and expect to see the following:

Assertion failed: (x=3) != (y=3), file sketch.ino, line 17.
Test bad failed.
Test ok passed.
Test summary: 1 passed, 1 failed, and 0 skipped, out of 2 test(s).

The following asserts are supported [with an optional footnote and return value]

| Assertion | Description | | --- | --- | | assertEqual(a,b [,footnote [,retval]]) | a == b? | | assertNear(a,b,maxerr, [,footnote[,retval]]) | abs(b-a)<=maxerr? | | assertRelativelyNear(a,b,maxerr, [,footnote[,retval]]) | abs(b-a)/abs(½(abs(a)+abs(b)))<=maxerr? | | assertNotEqual(a,b [,footnote[,retval]]) | a != b? | | assertLess(a,b [,footnote[,retval]]) | a < b? | | assertLessOrEqual(a,b [,footnote[,retval]]) | a <= b? | | assertMore(a,b [,footnote[,retval]]) | a > b? | | assertMoreOrEqual(a,b [,footnote[,retval]]) | a >= b? | | assertTrue(p [,footnote[,retval]]) | same as assertEqual(p,true) | | assertFalse(p [,footnote[,retval]]) | same as assertEqual(p,false) |

[,footnote[,retval]]

Assertions are replaced with essentially (retval is the optional last value of an assert):

if (not assertion) { fail(); return [retval]; }

When things go wrong, it can be useful to print additional information. As of 2.3.2-alpha, this is possible with any assertXXX() method by adding an additional third parameter [footnote] to the assert. For example,

test(cases)
{
  int x=3;
  for (int k=0; k<4; ++k) {
    assertNotEqual(x,k,"case k=" << k);
  }
}

will fail with the message

Assertion failed: (x=3) != (k=3), file basic.ino, line 20 [case k=3].

The additional message is only created if the assert actually needs to generate output (usually when it fails). It appears in the [] brackets at the end of the assert message. Notice you can create fairly complex messages by chaining things you can print (like Serial.print()) between << operators. This is similar to the C++ ostream insertion operators, if you are familar with that.

The status of the test can be used (bool ok) when printing the message. Under normal verbosity settings, ok will always be false, but more verbose settings can print assert messages even if they pass.

Selecting tests

In your setup() function, you can select which tests are going to be setup and looped. The default is that all tests are included.

Test::exclude(const char *pattern) removes all the tests that match the given *-pattern.

Test::include(const char *pattern) includes all the tests that match the given *-pattern.

Here are some examples:

Select examples:

A single test my_test

void setup()
{
  Serial.begin(9600);
  while(!Serial) {} // Portability for Leonardo/Micro
  Test::exclude("*");
  Test::include("my_test");
}

All tests named dev_-something, but not the ones ending in _skip or _slow, or have the word eeprom in them:

void setup()
{
  Serial.begin(9600);
  while(!Serial) {} // Portability for Leonardo/Micro
  Test::exclude("*");
  Test::include("dev_*");
  Test::exclude("*_slow");
  Test::exclude("*_skip");
  Test::exclude("*eeprom*");
}

Re-running tests (advanced)

Typically, you just want to run all tests once and then show the result. If so, you can skip this section.

In more advanced situations, you might want to run the entire test suite multiple times (for example if your tests can be configured with different parameters). To facilitate this, you can use the Test::resetDoneTests() function.

Calling this function will reset all completed tests (passed, failed or skipped) back to their initial state. For tests that define a setup method, this will be run again on the next Test::run(). If any tests were not completed yet, these are unaffected. The statistics (number of passed, failed and skipped tests) are also reset to 0.

Note that excluded tests (using Test::exclude()) are treated as skipped tests, so these are also re-run (you will need to call Test::exclude() again after resetDoneTests() if you want keep these tests excluded). Tests removed with their remove() method are really removed, so not re-run when using resetDoneTests().

Typically, you would use this after all tests are completed (but if you call resetDoneTests() when some tests are still pending, those tests are unaffected). You must never call resetDoneTests() from inside a test, only between calls to Test::run().

Below is an example that runs all tests once, then changes a global variable and runs all tests again. To have a bit more direct control over running the tests, this example does not call Test::run() infinitely in the loop(), but instead uses Test:runUntilDone() which repeatedly calls Test::run() until all tests are completed.

bool some_global_setting = false;

void setup() {
  Serial.begin(9600);
  while(!Serial) {} // Portability for Leonardo/Micro

  Test::runUntilDone();

  some_global_setting = true;
  Test::resetDoneTests();
  Test::runUntilDone();
}

void loop() { }

Output

The Test::out value is the shared value for all tests describing where output for all tests goes. The default is

Test::out = &Serial;

But you can set to the address of any Print stream. For example, if you want the output to be on the Serial3 device on the arduino mega, use

Test::out = &Serial3;

in your setup(). Note the library does not set the baud rate - you have to do that in your setup().

Verbosity

Normal ArduinoUnit verbosity reports only failed assertions, the status (pass,skip,fail) of completed tests, and a summary.

Seeing more.

It is often useful to see the results of assertions [and footnotes] even when they pass. If you want to trace everything in this way, you can turn on all output with Test::min_verbosity = TEST_VERBOSITY_ALL in your setup.

Seeing less more.

The previous choice is great until you are lost in an ocean of messages for tests you do not want to watch at the moment. Instead of globally setting min_verbosity/max_verbosity in your setup(), you can instead use verbosity = TEST_VERBOSITY_ALL in a given test to see everything about that test.

MockPrint and MockStream (intermediate)

MockPrint is provided by ArduinoUnit to mimic a real output device, like Serial, but is also a String which happens to contain the information printed to it. This can be used to test output formatting, as in:

void format(Print &out, int value) {
  out.print("decimal ");
  out.print(value);
  out.print(" is hex ");
  out.println(value,HEX);
}

test(format) {
  MockPrint mp;
  format(mp,32); // test as mock
  assertEqual(mp,"decimal 32 is hex 20\r\n");
}

void setup() {
  Serial.begin();
  while (!Serial) {}
  format(Serial,100); // format to serial
}

void loop() {
  Test::run();
}

MockStream is provided by ArduinoUnit to mimic a real input/output device, like Serial. It contains two MockPrint parts, input contains the input that will be read from the MockStream, and output which contains the output that was written. This can be used to test input and output, as in:

void square(Stream &io) {
  io.print("value? ");
  int x = io.parseInt();
  out.print(value);  
  out.print("*");
  out.print(value);
  out.print("=");
  out.println(x*x);
}

test(square) {
  MockStream ms;
  ms.input.print(10);
  square(ms);
  assertEqual(ms.output,"value? 10*10=100\r\n");
}

void setup() {
  Serial.begin();
  while (!Serial) {}
  square(Serial); // format to serial
}

void loop() {
  Test::run();
}

The mockstream example shows a convenient way to switch between real and mock streams for testing.

En Vitro Testing (advanced)

ArduinoUnit will compile in a standard C++ environment (LLVM or GCC) with -std=gnu++11. The advanced example has a makefile and main.cpp to support this.

Note ArduinoUnit has very limited mocking features; you will have to include the mocking features you need to simulate the embedded environment. The main.cpp file in the advanced example illustrates minimal mocking. In particular the only features provided (because of dependencies on these by ArduinoUnit) are:

F()
millis()
micros()
String
Print
Printable
Stream # public components only

These are available via #include "ArduinoUnitMock.h". In the mock environment, there are two additional objects, CppStreamPrint and CppIOStream, which wrap C++ std::ostream (and std::istream for CppIOStream). This simplifies creating tests in the mocking environments. Look at the advanced example and test firmware for guidance.

Verbosity (advanced)

Just how much information is generated on each test is fairly flexible, and designed to address these rules:

1. How much code is generated (TEST_MAX_VERBOSITY)
2. Global maximums subject to 1 (static Test::max_verbosity)
3. Global minimums subject to 1 and 2 (static Test::min_verbosity)
4. Per-test requirements subject to 1, 2 and 3 (Test::verbosity)

The rules are as follows for each kind of possibl