Perun: Lightweight Performance Version System

Perun is an open source light-weight Performance Version System, which works as a wrapper over projects in Version Control Systems (such as git) and (in parallel) tracks performance profiles (i.e., collection of performance metrics) corresponding to different versions of underlying project. Moreover, it offers a wide tool suite that can be used for automation of the performance regression test runs, postprocessing of resulting profiles (e.g., by creating performance models) or effective interpretation of the results.

In particular, Perun has the following advantages over, e.g., using databases or sole Version Control Systems for the same purpose:

Preserves Context---each performance profile is assigned to a concrete minor version adding the functional context (i.e., code changes) of profiles. This way, one knows precisely for which code version, the profile was collected.
Provides Automation---Perun allows one to easily automate the process of profile collection, eventually reducing the whole process to a single command. The specification of jobs is read from YAML files.
Is Highly Generic---supported format of the performance profiles is based on JSON. Perun tool suite contains a basic set of visualizations, postprocessing and collection modules, but it is easily extensible.
Is Easy to use---the workflow, interface and storage of Perun is heavily inspired by the git systems aiming at natural use.

Perun is intended to be used in two ways: (1) for a single developer (or a small team) as a complete solution for automating, storing and interpreting performance of project or (2) as a dedicated store for a bigger projects and teams. Its git-like design aims at easy distribution and simple interface makes it a good store of performance profiles along with the functional (or environmental) context.

Installation

[!WARNING] Installing Perun into the system Python environment may cause dependency conflicts with other packages. We highly suggest using some form of virtual environment to install and run Perun (see the Installation in Virtual Environment section).

You can install Perun in the system Python environment from pip as follows:

pip3 install perun-toolsuite

Alternatively you can install Perun from the source code as follows:

git clone https://github.com/Perfexionists/perun.git
cd perun
make install

These commands install Perun to your system as a runnable python package. You can then run Perun safely from the command line using the perun command. Run either perun --help or see the cli documentation for more information about running Perun commands from command line. Note that depending on your OS and the location of Python libraries, you might require root permissions to install Perun.

Installation in Virtual Environment

To avoid dependency clashes and other installation issues, we recommend installing Perun in some form of virtual environment. For example:

Using the venv standard library. This is the easiest default approach as no additional packages or dependencies are needed. For example, the following code snippet creates a virtual environment perun-venv in the current directory using the system Python version, activates the environment, and installs Perun in the environment.
```
python -m venv perun-venv
source perun-venv/bin/activate
pip install perun-toolsuite
```
It is also possible to create a virtual environment using different Python version, provided that the Python version executable is installed on the system. For example, python3.12 -m venv perun-venv will create a virtual environment for Python 3.12. Note that the perun CLI command will only be accessible when the perun-venv virtual environment is activated.
Using the pipx package. Pipx automatically creates new virtual environment for each installed Python application. Provided you have the pipx tool available on your system, Perun can be installed as follows:
```
pipx install perun-toolsuite
```
The perun CLI command will now be accessible directly without the need to activate any virtual environment.

Other forms of virtual environments might work as well, however, these are the only ones we tested.

System Dependencies

It is possible that some of the Perun commands may not be working at first, or that the installation fails (if some of the Python dependencies are not available as wheels and need to be built from source). This is likely due to some missing system dependencies:

g++
python-devel
libmagic
coreutils
cmake (complexity collector)
perf (kperf collector)
libunwind, libunwind-devel (memory collector)
perl-open (flamegraph view)

On macOS, additional system dependencies may be needed:

gnu-time
libunwind-headers

Limited support for macOS

Although Perun can be installed on macOS machines as well, most of the Perun collectors are not supported yet. Namely: bounds, complexity, kperf, memory and trace. Moreover, the perun import record is not supported as well due to the perf tool not being available on macOS.

Checking Perun Installation

It is advised to verify that Perun is running correctly in your environment as follows:

# Runs all tests using pytest
make test

or alternatively using Tox if you wish to test for more Python versions (see the developing section).

Note that we do not guarantee Perun to work on other Python versions than those marked as supported in the pyproject.toml file (also shown as a badge at the top of this README).

Installing Tracer Dependencies

If you wish to use our tracer profiler, you have to install its dependencies first. Please refer to INSTALL-tracer.md.

Developing

[!NOTE] Do not forget to check for missing system dependencies if the installation or some of the tests fail.

In order to commit changes to the Perun, you have to install Perun in development mode:

git clone https://github.com/Perfexionists/perun.git
cd perun
make dev

This method of installation allows you to make a changes to the code, which will be then reflected by the installation.

Again, it is advised to verify that Perun is running correctly in your environment as follows:

# Runs all tests using pytest
make test

Or you can use Tox to run tests for all the supported Python versions, as well as run static type checking, code style linting and generating documentation:

tox run

If you wish to test only against a single Python version, run Tox as:

tox run -e py3.Y

where Y is the python sub-version you wish to test. To see the available Tox environments (and consequently, the supported Python versions), run:

tox list

If you are interested in contributing to Perun project, please refer to contributing section. If you think your results could help others, please send us PR, we will review the code and in case it is suitable for wider audience, we will include it in our upstream.

But, please be understanding, we cannot fix and merge everything immediately.

Getting Started with Perun

In order to start managing performance of your project tracked by git, go to your project directory and run the following:

perun init --configure

This creates a parallel directory structure for Perun storage (stored in .perun), and creates initial configuration of the local project settings and opens it in the text editor (by default vim). The configuration is well commented and will help you with setting Perun up for your project. You can specify, e.g., the set of collectors (such as time or trace), or postprocessors and specify which commands (and with which arguments and workloads) should be profiled. See configuration for more details about Perun's configuration.

If you only wish to briefly try Perun out and assuming you have opened the configuration file from the previous step
(or you can open it from the path .perun/local.yaml), then we recommend to uncomment keys called `

Perun

Install / Use

README