Got Scripts? Test’em!

Unit testing in Javascript can be tedious and painful, but Testem makes it so easy that you will actually want to write tests.

Features

• Test-framework agnostic. Support for
  • Jasmine
  • QUnit
  • Mocha
  • Others, through custom test framework adapters.
• Run tests in all major browsers as well as Node and PhantomJS
• Two distinct use-cases:
  • Test-Driven-Development(TDD) — designed to streamline the TDD workflow
  • Continuous Integration(CI) — designed to work well with popular CI servers like Jenkins or Teamcity
• Cross-platform support
  • OS X
  • Windows
  • Linux
• Preprocessor support
  • CoffeeScript
  • Browserify
  • JSHint/JSLint
  • everything else

Screencasts

• Watch this introductory screencast (11:39) to see it in action! This one demonstrates the TDD workflow.
• Launchers (12:10) — more detail about launchers: how to specify what to auto-launch and how to configure one yourself to run tests in Node.
• Continuous Integration (CI) Mode (4:24) — details about how CI mode works.
• Making JavaScript Testing Fun With Testem (22:53) — a thorough screencast by NetTuts+'s Jeffery Way covering the basics, Jasmine, Mocha/Chai, CoffeeScript and more!

Installation

You need Node version 0.10+ or iojs installed on your system. Node is extremely easy to install and has a small footprint, and is really awesome otherwise too, so just do it.

Once you have Node installed:

npm install testem -g

This will install the testem executable globally on your system.

Usage

As stated before, Testem supports two use cases: test-driven-development and continuous integration. Let's go over each one.

Development Mode

The simplest way to use Testem, in the TDD spirit, is to start in an empty directory and run the command

testem

You will see a terminal-based interface which looks like this

Now open your browser and go to the specified URL. You should now see

We see 0/0 for tests because at this point we haven't written any code. As we write them, Testem will pick up any .js files that were added, include them, and if there are tests, run them automatically. So let's first write hello_spec.js in the spirit of "test first" (written in Jasmine)

describe('hello', function(){
  it('should say hello', function(){
    expect(hello()).toBe('hello world');
  });
});

Save that file and now you should see

Testem should automatically pick up the new files you've added and also any changes that you make to them and rerun the tests. The test fails as we'd expect. Now we implement the spec like so in hello.js

function hello(){
  return "hello world";
}

So you should now see

Using the Text User Interface

In development mode, Testem has a text-based graphical user interface which uses keyboard-based controls. Here is a list of the control keys

• ENTER : Run the tests
• q : Quit
• ← LEFT ARROW : Move to the next browser tab on the left
• → RIGHT ARROW : Move to the next browser tab on the right
• TAB : switch the target text panel between the top and bottom halves of the split panel (if a split is present)
• ↑ UP ARROW : scroll up in the target text panel
• ↓ DOWN ARROW : scroll down in the target text panel
• SPACE : page down in the target text panel
• b : page up in the target text panel
• d : half a page down target text panel
• u : half a page up target text panel

Command line options

To see all command line options

testem --help

Continuous Integration Mode

To use Testem for continuous integration

testem ci

In CI mode, Testem runs your tests on all the browsers that are available on the system one after another.

You can run multiple browsers in parallel in CI mode by specifying the --parallel (or -P) option to be the number of concurrent running browsers.

testem ci -P 5 # run 5 browser in parallel

To find out what browsers are currently available - those that Testem knows about and can make use of

testem launchers

Will print them out. The output might look like

$ testem launchers
Browsers available on this system:
IE7
IE8
IE9
Chrome
Firefox
Safari
Safari Technology Preview
Opera
PhantomJS

Did you notice that this system has IE versions 7-9? Yes, actually it has only IE9 installed, but Testem uses IE's compatibility mode feature to emulate IE 7 and 8.

When you run testem ci to run tests, it outputs the results in the TAP format by default, which looks like

ok 1 Chrome 16.0 - hello should say hello.

1..1
# tests 1
# pass  1

# ok

TAP is a human-readable and language-agnostic test result format. TAP plugins exist for popular CI servers

• Jenkins TAP plugin - I've added detailed instructions for setup with Jenkins.
• TeamCity TAP plugin

TAP Options

By default, the TAP reporter outputs all test results to the console, whether pass or fail. You can disable this behavior in order to make it easier to see which tests fail (i.e. only output failing tests) using:

{
  "tap_failed_tests_only": true
}

By default, the TAP reporter outputs console logs (distinct from pass/fail information) from all tests that emit logs to the console. You can disable this behavior and only emit logs for failed tests using:

{
  "tap_quiet_logs": true
}

For improved ergonomics, TAP reporter does not actually strictly adhere to the SPEC by default, reporting 'skip' as a possible status instead of as a directive. To strictly follow the spec use:

{
  "tap_strict_spec_compliance": true
}

By default, the TAP reporter outputs the result of JSON.stringify() for any log content that is not a String. You can override this behavior by specifying a function for tap_log_processor.

{
  "tap_log_processor": function(log) { return log.toString(); }
}

Other Test Reporters

Testem has other test reporters besides TAP: dot, xunit and teamcity. You can use the -R to specify them

testem ci -R dot

You can also add your own reporter.

Example xunit reporter output

Note that the real output is not pretty printed.

<testsuite name="Testem Tests" tests="4" failures="1" timestamp="Wed Apr 01 2015 11:56:20 GMT+0100 (GMT Daylight Time)" time="9">
  <testcase classname="PhantomJS 1.9" name="myFunc returns true when input is valid" time="0"/>
  <testcase classname="PhantomJS 1.9" name="myFunc returns false when user tickles it" time="0"/>
  <testcase classname="Chrome" name="myFunc returns true when input is valid" time="0"/>
  <testcase classname="Chrome" name="myFunc returns false when user tickles it" time="0">
    <failure name="myFunc returns false when user tickles it" message="function is not ticklish">
      <![CDATA[
      Callstack...
      ]]>
    </failure>
  </testcase>
</testsuite>

Example teamcity reporter output

##teamcity[testStarted name='PhantomJS 1.9 - hello should say hello']
##teamcity[testFinished name='PhantomJS 1.9 - hello should say hello']
##teamcity[testStarted name='PhantomJS 1.9 - hello should say hello to person']
##teamcity[testFinished name='PhantomJS 1.9 - hello should say hello to person']
##teamcity[testStarted name='PhantomJS 1.9 - goodbye should say goodbye']
##teamcity[testFailed name='PhantomJS 1.9 - goodbye should say goodbye' message='expected |'hello world|' to equal |'goodbye world|'' details='AssertionError: expected |'hello world|' to equal |'goodbye world|'|n    at http://localhost:7357/testem/chai.js:873|n    at assertEqual (http://localhost:7357/testem/chai.js:1386)|n    at http://localhost:7357/testem/chai.js:3627|n    at http://localhost:7357/hello_spec.js:14|n    at callFn (http://localhost:7357/testem/mocha.js:4338)|n    at http://localhost:7357/testem/mocha.js:4331|n    at http://localhost:7357/testem/mocha.js:4728|n    at http://localhost:7357/testem/mocha.js:4819|n    at next (http://localhost:7357/testem/mocha.js:4653)|n    at http://localhost:7357/testem/mocha.js:4663|n    at next (http://localhost:7357/testem/mocha.js:4601)|n    at http://localhost:7357/testem/mocha.js:4630|n    at timeslice (http://localhost:7357/testem/mocha.js:5761)']
##teamcity[testFinished name='PhantomJS 1.9 - goodbye should say goodbye']

##teamcity[testSuiteFinished name='mocha.suite' duration='11091']

Command line options

To see all command line options for CI

testem ci --help

Configuration File

For the simplest JavaScript projects, the TDD workflow described above will work fine. There are times when you want to structure your sou