pyresttest

What Is It?
Status
Installation
Sample Test
Examples
Installation
How Do I Use It?
Other Goodies
Basic Test Set Syntax
Benchmarking?
- Metrics
- Benchmark report formats:
RPM-based installation
Project Policies
FAQ
Feedback and Contributions

What Is It?

A REST testing and API microbenchmarking tool
Tests are defined in basic YAML or JSON config files, no code needed
Minimal dependencies (pycurl, pyyaml, optionally future), making it easy to deploy on-server for smoketests/healthchecks
Supports generate/extract/validate mechanisms to create full test scenarios
Returns exit codes on failure, to slot into automated configuration management/orchestration tools (also supplies parseable logs)
Logic is written and extensible in Python

Status

NEW: Full Python 3 Support in Alpha - download it, 'pip install future' and give it a try!

Apache License, Version 2.0

Changelog shows the past and present, milestones show the future roadmap.

The changelog will also show features/fixes currently merged to the master branch but not released to PyPi yet (pending installation tests across platforms).

Installation

PyRestTest works on Linux or Mac with Python 2.6, 2.7, or 3.3+ (with module 'future' installed)

First we need to install package python-pycurl:

Ubuntu/Debian: (sudo) apt-get install python-pycurl
CentOS/RHEL: (sudo) yum install python-pycurl
Alpine: (sudo) apk add curl-dev
Mac: don't worry about it
Other platforms: unsupported. You may get it to work by installing pycurl & pyyaml manually. Also include 'future' for Python 3. No guarantees though. This is needed because the pycurl dependency may fail to install by pip. In very rare cases you may need to intall python-pyyaml if pip cannot install it correctly.

It is easy to install the latest release by pip: (sudo) pip install pyresttest (also install 'future' if on Python 3)

If pip isn't installed, we'll want to install it first: If that is not installed, we'll need to install it first:

Ubuntu/Debian: (sudo) apt-get install python-pip
CentOS/RHEL: (sudo) yum install python-pip
Mac OS X with homebrew: brew install python (it's included)
Or with just python installed: wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py

Releases occur every few months, if you want to use unreleased features, it's easy to install from source:

See the Change Log for feature status.

git clone https://github.com/svanoort/pyresttest.git
cd pyresttest
sudo python setup.py install

The master branch tracks the latest; it is unit tested, but less stable than the releases (the 'stable' branch tracks tested releases).

Troubleshooting Installation

Almost all installation issues are due to problems with PyCurl and PyCurl's native libcurl bindings. It is easy to check if PyCurl is installed correctly:

python -c 'import pycurl'

If this returns correctly, pycurl is installed, if you see an ImportError or similar, it isn't. You may also verify the pyyaml installation as well, since that can fail to install by pip in rare circumstances.

Error installing by pip

__main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory

This is caused by libcurl not being installed or recognized: first install pycurl using native packages as above. Alternately, try installing just the libcurl libraries:

On Ubuntu/Debian: sudo apt-get install libcurl4-openssl-dev
On CentOS/RHEL: yum install libcurl-devel

VirtualEnv installation

PyCurl should install by pip, but sometimes has issues with pycurl/libcurl. Manually copying in a working system pycurl installation may help:

cp /usr/lib/python2.7/dist-packages/pycurl* env/local/lib/python2.7/site-packages/

Sample Test

This will check that APIs accept operations, and will smoketest an application

---
- config:
    - testset: "Basic tests"
    - timeout: 100  # Increase timeout from the default 10 seconds
- test: 
    - name: "Basic get"
    - url: "/api/person/"
- test: 
    - name: "Get single person"
    - url: "/api/person/1/"
- test: 
    - name: "Delete a single person, verify that works"
    - url: "/api/person/1/"
    - method: 'DELETE'
- test: # create entity by PUT
    - name: "Create/update person"
    - url: "/api/person/1/"
    - method: "PUT"
    - body: '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
    - headers: {'Content-Type': 'application/json'}
    - validators:  # This is how we do more complex testing!
        - compare: {header: content-type, comparator: contains, expected:'json'}
        - compare: {jsonpath_mini: 'login', expected: 'gbaltar'}  # JSON extraction
        - compare: {raw_body:"", comparator:contains, expected: 'Baltar' }  # Tests on raw response
- test: # create entity by POST
    - name: "Create person"
    - url: "/api/person/"
    - method: "POST"
    - body: '{"first_name": "William","last_name": "Adama","login": "theadmiral"}'
    - headers: {Content-Type: application/json}

Examples

The Quickstart should be everyone's starting point
Here's a really good example for how to create a user and then do tests on it.
- This shows how to use extraction from responses, templating, and different test types
If you're trying to do something fancy, take a look at the content-test.yaml.
- This shows most kinds of templating & variable uses. It shows how to read from file, using a variable in the file path, and templating on its content!
PyRestTest isn't limited to JSON; there's an example for submitting form data
There's a whole folder of example tests to help get started

How Do I Use It?

The Quickstart walks through common use cases
Benchmarking has its own section below
Advanced features have separate documentation (templating, generators, content extraction, complex validation).
How to extend PyRestTest is its own document
There are a ton of examples
@BastienAr has created an Atom editor package for PyRestTest development (thank you!)

Running A Simple Test

Run a basic test of the github API:

pyresttest https://api.github.com examples/github_api_smoketest.yaml

Using JSON Validation

A simple set of tests that show how json validation can be used to check contents of a response. Test includes both successful and unsuccessful validation using github API.

pyresttest https://api.github.com examples/github_api_test.yaml

(For help: pyresttest --help )

Interactive Mode

Same as the other test but running in interactive mode.

pyresttest https://api.github.com examples/github_api_test.yaml --interactive true --print-bodies true

Verbose Output

pyresttest https://api.github.com examples/github_api_test.yaml --log debug

Other Goodies

Simple templating of HTTP request bodies, URLs, and validators, with user variables
Generators to create dummy data for testing, with support for easily writing your own
Sequential tests: extract info from one test to use in the next
Import test sets in other test sets, to compose suites of tests easily
Easy benchmarking: convert any test to a benchmark, by changing the element type and setting output options if needed
Lightweight benchmarking: ~0.3 ms of overhead per request, and plans to reduce that in the future
Accurate benchmarking: network measurements come from native code in LibCurl, so test overhead doesn't alter them
Optional interactive mode for debugging and demos

Basic Test Set Syntax

As you can see, tests are defined in YAML format.

There are 5 top level test syntax elements:

url: a simple test, fetches given url via GET request and checks for good response code
test: a fully defined test (see below)
benchmark: a fully defined benchmark (see below)
config or configuration: overall test configuration (timeout is the most common option)
import: import another test set file so you Don't Repeat Yourself

Import example

---
# Will load the test sets from miniapp-test.yaml and run them
# Note that this will run AFTER the current test set

Pyresttest

Install / Use

README