Rove

Rove is a unit testing framework for Common Lisp applications. This is intended to be a successor of Prove.

Warning

This software is still BETA quality. The APIs will be likely to change.

Differences from Prove

• Supports ASDF package-inferred-system
• Fewer dependencies (Only depends on Trivial-Gray-Streams and UIOP)
• Reports details of failure tests
• Thread-support
• Allows setup/teardown and before/after hooks

Slides at Kansai Lisp #3

Usage

(defpackage jsonrpc/tests/request
  (:use :cl
        :rove
        :jsonrpc/request))
(in-package :jsonrpc/tests/request)

(deftest parse-message-test
  (testing "invalid message"
    (ok (signals (parse-message "xxx") 'jsonrpc-parse-error)
        "Parse error")))

$ rove tests/request.lisp
$ rove -r spec tests/request.lisp

Installation

(ql:quickload :rove)

Optionally, install the binary with Roswell:

$ ros install rove

API Reference

ok (form &optional description)

(ok (= a 10))
;-> ✓ Expect (= A 10) to be true.

(ok (= a 0))
;-> × 0) Expect (= A 0) to be true.

ng (form &optional description)

(ng (= a 10))
;-> × 0) Expect (= A 10) to be false.

(ng (= a 0))
;-> ✓ Expect (= A 0) to be false.

signals (form &optional condition)

(ok (signals (raise-error-form)))

outputs (form content &optional stream)

(ok (outputs (write-string "a") "a"))
;-> ✓ Expect (OUTPUTS (WRITE-STRING a) a) to be true.

expands (form expanded-form &optional env)

(defmacro defun-addn (n)
  (let ((m (gensym "m")))
    `(defun ,(intern (format nil "ADD~A" n)) (,m)
       (+ ,m ,n))))

(ok (expands '(defun-addn 10)
             `(defun add10 (#:m)
                (+ #:m 10))))

pass (description)

(pass "Okay. It's passed")
;-> ✓ Okay. It's passed

fail (description)

(fail "Oops. It's failed")
;-> × Oops. It's failed

skip (description)

(skip "Skipped for now.")
;-> - Skipped for now.

deftest (name &body body)

(deftest testing-length
  (ok (= (length #(1 2 3)) 3)))

testing (description &body body)

(deftest testing-length
  (testing "array"
    (ok (= (length #(1 2 3)) 3)))

  (testing "list"
    (ok (= (length (list 1 2 3)) 3))))

failing (description &body body)

Expects all assertions within the body to fail. If any assertion passes, the test fails and reports which assertions unexpectedly passed. This is useful for testing code that is known to be broken or not yet implemented.

(deftest incomplete-feature
  (failing "TODO: implement addition"
    (ok (= (add 1 2) 3))
    (ok (= (add 5 5) 10))))
;-> ✓ Expect (= (ADD 1 2) 3) to be true.
;-> ✓ Expect (= (ADD 5 5) 10) to be true.
;
; Summary shows:
; 0) INCOMPLETE-FEATURE
;      › TODO: implement addition
;    Expected to fail: Expect (= (ADD 1 2) 3) to be true.

setup (&body body)

Evaluates before testing the package once. This would be useful for initialization of tests, like establishment to the DB or creating a temporary directory.

Use (defhook :before ...) instead for running before each test.

(setup
  (ensure-directories-exist *tmp-directory*))

teardown (&body body)

Evaluates after testing the package once. This is executed even when the test is interrupted with some reason. This would be useful for cleanup of tests, like disconnecting from DB or deleting a temporary directory.

Use (defhook :after ...) instead for running after each test.

(teardown
  (uiop:delete-directory-tree *tmp-directory* validate t :if-does-not-exist :ignore))

defhook (name mode &body body)

Evaluates before/after running a each test in the package.

(defhook my-db-hook :before
  ...)

run (package &key style env)

(run :myapp/tests)                              ; Defaults to spec-style output.
(run :myapp/tests :style :spec)                 ; Detailed test output.
(run :myapp/tests :style :dot)                  ; One-dot-per-test output.
(run :myapp/tests :style :none)                 ; Minimal test output with filenames only.
(run :myapp/tests :env '(("APP_ENV" . "test")))

run* (package-pattern &key style env)

Same as run except it takes a pattern for test package names.

(run* "myapp/tests/*")

Coverage report

Coverage report can be enabled by setting the environment variable COVERAGE other than an empty string. This feature is an experimental and supported only for SBCL (The option is just ignored for other implementations).

$ COVERAGE=1 rove example.asd

FAQ

Q. How to run tests when loading files?

Add (rove:run-suite *package*) at the end of test files.

(defpackage #:rove-example
  (:use #:cl
        #:rove))
(in-package #:rove-example)

(deftest example-test
  (ok (= 1 1)))

(run-suite *package*)

Examples

• fukamachi/sanitized-params
• fukamachi/jsonrpc (package-inferred-system)

Portability

Developed for SBCL and tested successfully with:

• ABCL 1.5.0 and 1.6.0-dev
• CLISP 2.49.92
• ClozureCL 1.11.5
• ECL 16.1.3

Author

• Eitaro Fukamachi (e.arrows@gmail.com)

Copyright

Copyright (c) 2017 Eitaro Fukamachi

License

Licensed under the BSD 3-Clause License.