JenkinsPipelineUnit Testing Framework

Jenkins Pipeline Unit is a testing framework for unit testing Jenkins pipelines, written in Groovy Pipeline DSL.

If you use Jenkins as your CI workhorse (like us @ lesfurets.com) and you enjoy writing pipeline-as-code, you already know that pipeline code is very powerful but can get pretty complex.

This testing framework lets you write unit tests on the configuration and conditional logic of the pipeline code, by providing a mock execution of the pipeline. You can mock built-in Jenkins commands, job configurations, see the stacktrace of the whole execution and even track regressions.

Table of Contents

1. Usage
2. Configuration
3. Declarative Pipeline
4. Testing Shared Libraries
5. Writing Testable Libraries
6. Note On CPS
7. Contributing
8. Demos and Examples

Usage

Add to Your Project as Test Dependency

JenkinsPipelineUnit requires Java 17, since this is also the minimum version required by some library dependencies. Also note that JenkinsPipelineUnit is not currently compatible with Groovy 4, please see this issue for more details.

Note: Starting from version 1.2, artifacts are published to https://repo.jenkins-ci.org/releases.

Maven

<repositories>
    <repository>
    <id>jenkins-ci-releases</id>
    <url>https://repo.jenkins-ci.org/releases/</url>
    </repository>
    ...
</repositories>

<dependencies>
    <dependency>
        <groupId>com.lesfurets</groupId>
        <artifactId>jenkins-pipeline-unit</artifactId>
        <version>1.9</version>
        <scope>test</scope>
    </dependency>
    ...
</dependencies>

Gradle

repositories {
    maven { url 'https://repo.jenkins-ci.org/releases/' }
    ...
}

dependencies {
    testImplementation "com.lesfurets:jenkins-pipeline-unit:1.9"
    ...
}

Start Writing Tests

You can write your tests in Groovy or Java, using the test framework you prefer. The easiest entry point is extending the abstract class BasePipelineTest, which initializes the framework with JUnit.

Let's say you wrote this awesome pipeline script, which builds and tests your project:

def execute() {
    node() {
        String utils = load 'src/test/jenkins/lib/utils.jenkins'
        String revision = stage('Checkout') {
            checkout scm
            return utils.currentRevision()
        }
        gitlabBuilds(builds: ['build', 'test']) {
            stage('build') {
                gitlabCommitStatus('build') {
                    sh "mvn clean package -DskipTests -DgitRevision=$revision"
                }
            }

            stage('test') {
                gitlabCommitStatus('test') {
                    sh "mvn verify -DgitRevision=$revision"
                }
            }
        }
    }
}

return this

Now using the Jenkins Pipeline Unit you can write a unit test to see if it does the job:

import com.lesfurets.jenkins.unit.BasePipelineTest

class TestExampleJob extends BasePipelineTest {
    @Test
    void shouldExecuteWithoutErrors() {
        loadScript('job/exampleJob.jenkins').execute()
        printCallStack()
    }
}

This test will print the call stack of the execution, which should look like so:

   exampleJob.run()
   exampleJob.execute()
      exampleJob.node(groovy.lang.Closure)
         exampleJob.load(src/test/jenkins/lib/utils.jenkins)
            utils.run()
         exampleJob.stage(Checkout, groovy.lang.Closure)
            exampleJob.checkout({$class=GitSCM, branches=[{name=feature_test}], extensions=[], userRemoteConfigs=[{credentialsId=gitlab_git_ssh, url=github.com/lesfurets/JenkinsPipelineUnit.git}]})
            utils.currentRevision()
               utils.sh({returnStdout=true, script=git rev-parse HEAD})
         exampleJob.gitlabBuilds({builds=[build, test]}, groovy.lang.Closure)
            exampleJob.stage(build, groovy.lang.Closure)
               exampleJob.gitlabCommitStatus(build, groovy.lang.Closure)
                  exampleJob.sh(mvn clean package -DskipTests -DgitRevision=bcc19744)
            exampleJob.stage(test, groovy.lang.Closure)
               exampleJob.gitlabCommitStatus(test, groovy.lang.Closure)
                  exampleJob.sh(mvn verify -DgitRevision=bcc19744)

Mocking Jenkins Variables

You can define both environment variables and job execution parameters.

import com.lesfurets.jenkins.unit.BasePipelineTest

class TestExampleJob extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        super.setUp()
        // Assigns false to a job parameter ENABLE_TEST_STAGE
        addParam('ENABLE_TEST_STAGE', 'false')
        // Assigns 1.0.0-rc.1 to the environment variable TAG_NAME
        addEnvVar('TAG_NAME', '1.0.0-rc.1')
        // Defines the previous execution status
        binding.getVariable('currentBuild').previousBuild = [result: 'UNSTABLE']
    }

    @Test
    void verifyParam() {
        assertEquals('false', binding.getVariable('params')['ENABLE_TEST_STAGE'])
    }
}

After calling super.setUp(), the test helper instance is available, as well as many helper methods. The test helper already provides basic variables such as a very simple currentBuild definition. You can redefine them as you wish.

Note that super.setUp() must be called prior to using most features. This is commonly done using your own setUp method, decorated with @Override and @BeforeEach.

Parameters added via addParam are immutable, which reflects the same behavior in Jenkins. Attempting to modify the params map in the binding will result in an error.

Mocking Jenkins Commands

You can register interceptors to mock pipeline methods, including Jenkins commands, which may or may not return a result.

import com.lesfurets.jenkins.unit.BasePipelineTest

class TestExampleJob extends BasePipelineTest {
    @Override
    @BeforeEach
    void setUp() {
        super.setUp()
        helper.registerAllowedMethod('sh', [Map]) { args -> return 'bcc19744' }
        helper.registerAllowedMethod('timeout', [Map, Closure], null)
        helper.registerAllowedMethod('timestamps', []) { println 'Printing timestamp' }
        helper.registerAllowedMethod('myMethod', [String, int]) { String s, int i ->
            println "Executing myMethod mock with args: '${s}', '${i}'"
        }
    }
}

The test helper already includes mocks for all base pipeline steps as well as a steps from a few widely-used plugins. You need to register allowed methods if you want to override these mocks and add others. Note that you need to provide a method signature and a callback (closure or lambda) in order to allow a method. Any method call which is not recognized will throw an exception.

Please refer to the BasePipelineTest class for the list of currently supported mocks.

Some tricky methods such as load and parallel are implemented directly in the helper. If you want to override those, make sure that you extend the PipelineTestHelper class.

Mocking readFile and fileExists

The readFile and fileExists steps can be mocked to return a specific result for a given file name. This can be useful for testing pipelines for which file operations can influence subsequent steps. An example of such a testing scenario follows:

// Jenkinsfile
node {
    stage('Process output') {
        if (fileExists('output') && readFile('output') == 'FAILED!!!') {
            currentBuild.result = 'FAILURE'
            error 'Build failed'
        }
    }
}

@Test
void exampleReadFileTest() {
    helper.addFileExistsMock('output', true)
    helper.addReadFileMock('output', 'FAILED!!!')

    runScript('Jenkinsfile')

    assertJobStatusFailure()
}

Mocking Shell Steps

The shell steps (sh, bat, etc) are used by many pipelines for a variety of tasks. They can be mocked to either (a) statically return:

• A string for standard output
• A return code

Or (b), to execute a closure that returns a Map (with stdout and exitValue entries). The closure will be executed when the shell is called, allowing for dynamic behavior.

Here is a sample pipeline and corresponding unit tests for each of these variants.

// Jenkinsfile
node {
    stage('Mock build') {
        String systemType = sh(returnStdout: true, script: 'uname')
        if (systemType == 'Debian') {
            sh './build.sh --release'
            int status = sh(returnStatus: true, script: './test.sh')
            if (status > 0) {
                currentBuild.result = 'UNSTABLE'
            } else {
                def result = sh(
                    returnStdout: true,
                    script: './processTestResults.sh --platform debian',
                )
                if (!result.endsWith('SUCCESS')) {
                    currentBuild.result = 'FAILURE'
                    error 'Build failed!'