Parameterized testing with any Python test framework

.. image:: https://img.shields.io/pypi/v/parameterized :alt: PyPI :target: https://pypi.org/project/parameterized/

.. image:: https://img.shields.io/pypi/dm/parameterized :alt: PyPI - Downloads :target: https://pypi.org/project/parameterized/

.. image:: https://circleci.com/gh/wolever/parameterized.svg?style=svg :alt: Circle CI :target: https://circleci.com/gh/wolever/parameterized

Parameterized testing in Python sucks.

parameterized fixes that. For everything. Parameterized testing for nose, parameterized testing for py.test, parameterized testing for unittest.

.. code:: python

test_math.py

from nose.tools import assert_equal from parameterized import parameterized, parameterized_class

import unittest import math

@parameterized([ (2, 2, 4), (2, 3, 8), (1, 9, 1), (0, 9, 0), ]) def test_pow(base, exponent, expected): assert_equal(math.pow(base, exponent), expected)

class TestMathUnitTest(unittest.TestCase): @parameterized.expand([ ("negative", -1.5, -2.0), ("integer", 1, 1.0), ("large fraction", 1.6, 1), ]) def test_floor(self, name, input, expected): assert_equal(math.floor(input), expected)

@parameterized_class(('a', 'b', 'expected_sum', 'expected_product'), [ (1, 2, 3, 2), (5, 5, 10, 25), ]) class TestMathClass(unittest.TestCase): def test_add(self): assert_equal(self.a + self.b, self.expected_sum)

  def test_multiply(self):
     assert_equal(self.a * self.b, self.expected_product)

@parameterized_class([ { "a": 3, "expected": 2 }, { "b": 5, "expected": -4 }, ]) class TestMathClassDict(unittest.TestCase): a = 1 b = 1

  def test_subtract(self):
     assert_equal(self.a - self.b, self.expected)

With nose (and nose2)::

$ nosetests -v test_math.py
test_floor_0_negative (test_math.TestMathUnitTest) ... ok
test_floor_1_integer (test_math.TestMathUnitTest) ... ok
test_floor_2_large_fraction (test_math.TestMathUnitTest) ... ok
test_math.test_pow(2, 2, 4, {}) ... ok
test_math.test_pow(2, 3, 8, {}) ... ok
test_math.test_pow(1, 9, 1, {}) ... ok
test_math.test_pow(0, 9, 0, {}) ... ok
test_add (test_math.TestMathClass_0) ... ok
test_multiply (test_math.TestMathClass_0) ... ok
test_add (test_math.TestMathClass_1) ... ok
test_multiply (test_math.TestMathClass_1) ... ok
test_subtract (test_math.TestMathClassDict_0) ... ok

----------------------------------------------------------------------
Ran 12 tests in 0.015s

OK

As the package name suggests, nose is best supported and will be used for all further examples.

With py.test (version 2.0 and above)::

$ py.test -v test_math.py
============================= test session starts ==============================
platform darwin -- Python 3.6.1, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
collecting ... collected 13 items

test_math.py::test_pow::[0] PASSED
test_math.py::test_pow::[1] PASSED
test_math.py::test_pow::[2] PASSED
test_math.py::test_pow::[3] PASSED
test_math.py::TestMathUnitTest::test_floor_0_negative PASSED
test_math.py::TestMathUnitTest::test_floor_1_integer PASSED
test_math.py::TestMathUnitTest::test_floor_2_large_fraction PASSED
test_math.py::TestMathClass_0::test_add PASSED
test_math.py::TestMathClass_0::test_multiply PASSED
test_math.py::TestMathClass_1::test_add PASSED
test_math.py::TestMathClass_1::test_multiply PASSED
test_math.py::TestMathClassDict_0::test_subtract PASSED
==================== 12 passed, 4 warnings in 0.16 seconds =====================

With unittest (and unittest2)::

$ python -m unittest -v test_math
test_floor_0_negative (test_math.TestMathUnitTest) ... ok
test_floor_1_integer (test_math.TestMathUnitTest) ... ok
test_floor_2_large_fraction (test_math.TestMathUnitTest) ... ok
test_add (test_math.TestMathClass_0) ... ok
test_multiply (test_math.TestMathClass_0) ... ok
test_add (test_math.TestMathClass_1) ... ok
test_multiply (test_math.TestMathClass_1) ... ok
test_subtract (test_math.TestMathClassDict_0) ... ok

----------------------------------------------------------------------
Ran 8 tests in 0.001s

OK

(note: because unittest does not support test decorators, only tests created with @parameterized.expand will be executed)

With green::

$ green test_math.py -vvv
test_math
  TestMathClass_1
.   test_method_a
.   test_method_b
  TestMathClass_2
.   test_method_a
.   test_method_b
  TestMathClass_3
.   test_method_a
.   test_method_b
  TestMathUnitTest
.   test_floor_0_negative
.   test_floor_1_integer
.   test_floor_2_large_fraction
  TestMathClass_0
.   test_add
.   test_multiply
  TestMathClass_1
.   test_add
.   test_multiply
  TestMathClassDict_0
.   test_subtract

Ran 12 tests in 0.121s

OK (passes=9)

Installation

::

$ pip install parameterized

Compatibility

Yes__ (mostly).

__ https://app.circleci.com/pipelines/github/wolever/parameterized?branch=master

.. list-table:: :header-rows: 1 :stub-columns: 1

• • Py3.7
  • Py3.8
  • Py3.9
  • Py3.10
  • Py3.11
  • PyPy3
  • @mock.patch
• • nose
  • yes
  • yes
  • yes
  • yes
  • no§
  • no§
  • yes
• • nose2
  • yes
  • yes
  • yes
  • yes
  • yes
  • yes
  • yes
• • py.test 2
  • no*
  • no*
  • no*
  • no*
  • no*
  • no*
  • no*
• • py.test 3
  • yes
  • yes
  • yes
  • yes
  • no*
  • no*
  • yes
• • py.test 4
  • no**
  • no**
  • no**
  • no**
  • no**
  • no**
  • no**
• • py.test fixtures
  • no†
  • no†
  • no†
  • no†
  • no†
  • no†
  • no†
• • | unittest | (@parameterized.expand)
  • yes
  • yes
  • yes
  • yes
  • yes
  • yes
  • yes
• • | unittest2 | (@parameterized.expand)
  • yes
  • yes
  • yes
  • yes
  • no§
  • no§
  • yes

§: nose and unittest2 - both of which were last updated in 2015 - sadly do not appear to support Python 3.10 or 3.11.

*: py.test 2 does not appear to work under Python 3 (#71), and py.test 3 does not appear to work under Python 3.10 or 3.11 (#154).

**: py.test 4 is not yet supported (but coming!) in issue #34__

†: py.test fixture support is documented in issue #81__

__ https://github.com/wolever/parameterized/issues/71 __ https://github.com/wolever/parameterized/issues/154 __ https://github.com/wolever/parameterized/issues/34 __ https://github.com/wolever/parameterized/issues/81

Dependencies

(this section left intentionally blank)

Exhaustive Usage Examples

The @parameterized and @parameterized.expand decorators accept a list or iterable of tuples or param(...), or a callable which returns a list or iterable:

.. code:: python

from parameterized import parameterized, param

# A list of tuples
@parameterized([
    (2, 3, 5),
    (3, 5, 8),
])
def test_add(a, b, expected):
    assert_equal(a + b, expected)

# A list of params
@parameterized([
    param("10", 10),
    param("10", 16, base=16),
])
def test_int(str_val, expected, base=10):
    assert_equal(int(str_val, base=base), expected)

# An iterable of params
@parameterized(
    param.explicit(*json.loads(line))
    for line in open("testcases.jsons")
)
def test_from_json_file(...):
    ...

# A callable which returns a list of tuples
def load_test_cases():
    return [
        ("test1", ),
        ("test2", ),
    ]
@parameterized(load_test_cases)
def test_from_function(name):
    ...

.. **

Note that, when using an iterator or a generator, all the items will be loaded into memory before the start of the test run (we do this explicitly to ensure that generators are exhausted exactly once in multi-process or multi-threaded testing environments).

The @parameterized decorator can be used test class methods, and standalone functions:

.. code:: python

from parameterized import parameterized

class AddTest(object):
    @parameterized([
        (2, 3, 5),
    ])
    def test_add(self, a, b, expected):
        assert_equal(a + b, expected)

@parameterized([
    (2, 3, 5),
])
def test_add(a, b, expected):
    assert_equal(a + b, expected)

And @parameterized.expand can be used to generate test methods in situations where test generators cannot be used (for example, when the test class is a subclass of unittest.TestCase):

.. code:: python

import unittest
from parameterized import parameterized

class AddTestCase(unittest.TestCase):
    @parameterized.expand([
        ("2 and 3", 2, 3, 5),
        ("3 and 5", 3, 5, 8),
    ])
    def test_add(self, _, a, b, expected):
        assert_equal(a + b, expected)

Will create the test cases::

$ nosetests example.py
test_add_0_2_and_3 (example.AddTestCase) ... ok
test_add_1_3_and_5 (example.AddTestCase) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Note that @parameterized.expand works by creating new methods on the test class. If the first parameter is a string, that string will be added to the end of the method name. For example, the test case above will generate the methods test_add_0_2_and_3 and test_add_1_3_and_5.

The names of the test cases generated b