############################################################################################# cwltool: The reference implementation of the Common Workflow Language standards #############################################################################################

|Linux Status| |Coverage Status| |Docs Status|

PyPI: |PyPI Version| |PyPI Downloads Month| |Total PyPI Downloads|

Conda: |Conda Version| |Conda Installs|

Debian: |Debian Testing package| |Debian Stable package|

Quay.io (Docker): |Quay.io Container|

.. |Linux Status| image:: https://github.com/common-workflow-language/cwltool/actions/workflows/ci-tests.yml/badge.svg?branch=main :target: https://github.com/common-workflow-language/cwltool/actions/workflows/ci-tests.yml

.. |Debian Stable package| image:: https://badges.debian.net/badges/debian/stable/cwltool/version.svg :target: https://packages.debian.org/stable/cwltool

.. |Debian Testing package| image:: https://badges.debian.net/badges/debian/testing/cwltool/version.svg :target: https://packages.debian.org/testing/cwltool

.. |Coverage Status| image:: https://img.shields.io/codecov/c/github/common-workflow-language/cwltool.svg :target: https://codecov.io/gh/common-workflow-language/cwltool

.. |PyPI Version| image:: https://badge.fury.io/py/cwltool.svg :target: https://badge.fury.io/py/cwltool

.. |PyPI Downloads Month| image:: https://pepy.tech/badge/cwltool/month :target: https://pepy.tech/project/cwltool

.. |Total PyPI Downloads| image:: https://static.pepy.tech/personalized-badge/cwltool?period=total&units=international_system&left_color=black&right_color=orange&left_text=Total%20PyPI%20Downloads :target: https://pepy.tech/project/cwltool

.. |Conda Version| image:: https://anaconda.org/conda-forge/cwltool/badges/version.svg :target: https://anaconda.org/conda-forge/cwltool

.. |Conda Installs| image:: https://anaconda.org/conda-forge/cwltool/badges/downloads.svg :target: https://anaconda.org/conda-forge/cwltool

.. |Quay.io Container| image:: https://quay.io/repository/commonwl/cwltool/status :target: https://quay.io/repository/commonwl/cwltool

.. |Docs Status| image:: https://readthedocs.org/projects/cwltool/badge/?version=latest :target: https://cwltool.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status

This is the reference implementation of the Common Workflow Language open standards <https://www.commonwl.org/>_. It is intended to be feature complete and provide comprehensive validation of CWL files as well as provide other tools related to working with CWL.

cwltool is written and tested for Python <https://www.python.org/>_ 3.x {x = 9, 10, 11, 12, 13}

The reference implementation consists of two packages. The cwltool package is the primary Python module containing the reference implementation in the cwltool module and console executable by the same name.

The cwlref-runner package is optional and provides an additional entry point under the alias cwl-runner, which is the implementation-agnostic name for the default CWL interpreter installed on a host.

cwltool is provided by the CWL project, a member project of Software Freedom Conservancy <https://sfconservancy.org/news/2018/apr/11/cwl-new-member-project/>_ and our many contributors <https://github.com/common-workflow-language/cwltool/graphs/contributors>_.

.. contents:: Table of Contents

---

Install

---

cwltool packages

Your operating system may offer cwltool directly. For Debian <https://tracker.debian.org/pkg/cwltool>, Ubuntu <https://launchpad.net/ubuntu/+source/cwltool>, and similar Linux distribution try

.. code:: bash

sudo apt-get install cwltool

If you encounter an error, first try to update package information by using

.. code:: bash

sudo apt-get update

If you are running macOS X or other UNIXes and you want to use packages prepared by the conda-forge project, then please follow the install instructions for conda-forge <https://conda-forge.org/#about>_ (if you haven't already) and then

.. code:: bash

conda install -c conda-forge cwltool

All of the above methods of installing cwltool use packages that might contain bugs already fixed in newer versions or be missing desired features. If the packaged version of cwltool available to you is too old, then we recommend installing using pip and venv

.. code:: bash

python3 -m venv env # Create a virtual environment named 'env' in the current directory source env/bin/activate # Activate environment before installing cwltool

Then install the latest cwlref-runner package from PyPi (which will install the latest cwltool package as well)

.. code:: bash

pip install cwlref-runner

If installing alongside another CWL implementation (like toil-cwl-runner or arvados-cwl-runner) then instead run

.. code:: bash

pip install cwltool

MS Windows users

1. Install Windows Subsystem for Linux 2 and Docker Desktop <https://docs.docker.com/docker-for-windows/wsl/#prerequisites>_.
2. Install Debian from the Microsoft Store <https://www.microsoft.com/en-us/p/debian/9msvkqc78pk6>_.
3. Set Debian as your default WSL 2 distro: wsl --set-default debian.
4. Return to the Docker Desktop, choose Settings → Resources → WSL Integration and under "Enable integration with additional distros" select "Debian",
5. Reboot if you have not yet already.
6. Launch Debian and follow the Linux instructions above (apt-get install cwltool or use the venv method)

Network problems from within WSL2? Try these instructions <https://github.com/microsoft/WSL/issues/4731#issuecomment-702176954>_ followed by wsl --shutdown.

cwltool development version

Or you can skip the direct pip commands above and install the latest development version of cwltool:

.. code:: bash

git clone https://github.com/common-workflow-language/cwltool.git # clone (copy) the cwltool git repository cd cwltool # Change to source directory that git clone just downloaded pip install .[deps] # Installs cwltool from source cwltool --version # Check if the installation works correctly

Remember, if co-installing multiple CWL implementations, then you need to maintain which implementation cwl-runner points to via a symbolic file system link or another facility <https://wiki.debian.org/DebianAlternatives>_.

Recommended Software

We strongly suggested to have the following installed:

• One of the following software container engines

  • Podman <https://podman.io/getting-started/installation>_
  • Docker <https://docs.docker.com/engine/install/>_
  • Singularity/Apptainer: See Using Singularity_
  • udocker: See Using uDocker_

• node.js <https://nodejs.org/en/download/>_ for evaluating CWL Expressions quickly (required for udocker users, optional but recommended for the other container engines).

Without these, some examples in the CWL tutorials at http://www.commonwl.org/user_guide/ may not work.

---

Run on the command line

---

Simple command::

cwl-runner my_workflow.cwl my_inputs.yaml

Or if you have multiple CWL implementations installed and you want to override the default cwl-runner then use::

cwltool my_workflow.cwl my_inputs.yml

You can set cwltool options in the environment with CWLTOOL_OPTIONS, these will be inserted at the beginning of the command line::

export CWLTOOL_OPTIONS="--debug"

Use with boot2docker on macOS

boot2docker runs Docker inside a virtual machine, and it only mounts Users on it. The default behavior of CWL is to create temporary directories under e.g. /Var which is not accessible to Docker containers.

To run CWL successfully with boot2docker you need to set the --tmpdir-prefix and --tmp-outdir-prefix to somewhere under /Users::

$ cwl-runner --tmp-outdir-prefix=/Users/username/project --tmpdir-prefix=/Users/username/project wc-tool.cwl wc-job.json

Docker default platform on macOS with Apple Silicon

If macOS users want to run CWL tools/workflows using cwltool with Docker and their software containers only have support for amd64 (64-bit x86) CPUs, but they have an Apple Silicon (aarch64/arm64) CPU, they run into the error:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested.

To fix this, export the following environment variable before executing cwltool:

export DOCKER_DEFAULT_PLATFORM=linux/amd64

To automatically have this variable set in the future, add the same command to ones respective shell profile (e.g. ~/.zshrc, ~/.bash_profile).

Using uDocker

Some shared computing environments don't support Docker software containers for technical or policy reasons. As a workaround, the CWL reference runner supports using the udocker <https://github.com/indigo-dc/udocker>_ program on Linux using --udocker.

udocker installation: https://indigo-dc.github.io/udocker/installation_manual.html

Run cwltool just as you usually would, but with --udocker prior to the workflow path:

.. code:: bash

cwltool --udocker https://github.com/common-workflow-language/common-workflow-language/raw/main/v1.0/v1.0/test-cwl-out2.cwl https://github.com/common-workflow-language/common-workflow-language/raw/main/v1.0/v1.0/empty.json

As was mentioned in the Recommended Software_ section,

Using Singularity

cwltool can also use Singularity <https://github.com/hpcng/singularity/releases/>_ version 2.6.1 or later as a Docker container runtime. cwltool with Singularity will run software containers specified in DockerRequirement and therefore wo