Tmm

ANNOUNCEMENT: TMM and tmm4py are now available on PyPI and can be installed in a single step that automatically builds and installs all their dependencies: pip install tmm4py. This was a nontrivial undertaking that would not have been possible without Jamie Carr's help.

ANNOUNCEMENT: I'm pleased to announce the release of version 3 of the Transport Matrix Method (TMM) software. Included in this release is tmm4py, a full-featured wrapper exposing all of the TMM's functionality in Python. tmm4py enables one to write biogeochemical models in Python, Fortran and/or C and run them entirely from Python. To make this possible, the TMM software has been redesigned from the ground up along object oriented principles as a library callable from other languages. A huge thank you to Jamie Carr, without whose help and expertise the tmm4py project would not have gotten off the ground.

You can read a description of tmm4py and the new TMM library in this J. Adv. Model. Earth Sys. article: http://dx.doi.org/10.1029/2025MS005028.

---

This is the Transport Matrix Method (TMM) code repository. It includes the core TMM software library, a Python wrapper tmm4py, and various biogeochemical models adapted to the TMM framework. The TMM software is written using the PETSc framework (http://www.mcs.anl.gov/petsc/).

How to cite:

This repository and all versions of the TMM and tmm4py software can be cited as: Khatiwala (2018; https://doi.org/10.5281/zenodo.1246299).

If you use the TMM, please cite Khatiwala et al. (2005; https://doi.org/10.1016/j.ocemod.2004.04.002) and Khatiwala (2007; https://doi.org/10.1029/2007GB002923). If you use tmm4py, please cite Khatiwala (2025; http://dx.doi.org/10.1029/2025MS005028). If you use any of my transport matrices or related data please cite this GitHub page (Khatiwala, 2018; https://doi.org/10.5281/zenodo.1246299) as the source, as well as Khatiwala (2007) and the papers for the relevant transport matrix configurations listed here: https://sites.google.com/view/samarkhatiwala-research-tmm. Thank you!

IMPORTANT: Please do NOT post the TMM code or any data files downloaded from this or related websites on your own github or other website. See LICENSE.txt for licensing information.

Feel free to email if you have any questions: samkat6@gmail.com

---

To install TMM/tmm4py and any required libraries you will need at a minimum need a C compiler. For tmm4py you will also need Python 3.9 and up. A Fortran compiler is required to run the Fortran examples.

In the following, it is assumed you're using bash as your shell. Change the commands appropriately if you're using a different shell (e.g.,change export to setenv, etc for tcsh).

Notes:

(1) The dependencies for TMM are: PETSc. The dependencies for tmm4py are: PETSc, petsc4py and TMM. PETSc in turn depends on MPI and BLAS/LAPACK. petsc4py and tmm4py also depend on NumPy and Cython.

(2) TMM and tmm4py are available as independent packages on PyPI for installation with pip. (TMM is called tmmlib as there was already another package called TMM.) If PETSc and petsc4py are not already available on your system, the installers will build and install them for you.

(3) On many HPC systems, optimized MPI and BLAS/LAPACK libraries will be already installed. PETSc may also be already installed and you just have to set the PETSC_DIR and PETSC_ARCH environment variables (see https://petsc.org/main/install/). While PETSc and petsc4py can be installed with pip, for production runs optimized for your hardware and software libraries it is recommended you build them from source (see instructions below). That said, as described below, even with pip you can control in detail how PETSc is configured and built by setting the PETSC_CONFIGURE_OPTIONS environment variable.

(4) If MPI is not already available, it is recommended that you install it either from source or using a package manager (conda, brew, etc). Basic instructions for this are provided below. You can also install MPI via pip, assuming binaries are available for your platform (they appear to be for the most common ones). However, as of now this doesn't install compiler wrappers for Fortran. So if you want to use models written in Fortran then you should install MPI from source or with a different package manager.

(5) MacOS users: The PETSc build system runs checks to ensure you have a functioning MPI. On MacOS, because of Apple's annoying security policies, this will throw up a dialog box asking you to allow this. Just click OK. If you don't, the checks will likely fail and, while PETSc will still install, it will switch off its MPI functionality. Bottomline is don't walk away from your computer when performing any of the installation steps below.

Installation using pip

Notes:

(i) In the following, if you only want to use the TMM and don't want to use Python/tmm4py, then replace pip install tmm4py with pip install tmmlib . You can also first install tmmlib with pip install tmmlib, set its path with:

export TMM_DIR=`python3 -c "from tmmlib import get_tmm_dir; print(get_tmm_dir())"`

and then follow the instructions below for installing tmm4py.

(ii) It is highly recommended you do a pip cache purge before you run the install commands below.

(iii) If you run into issues installing with pip add "-v" to the 'pip install xxx' commands below and email me the full output.

(iv) When installing PETSc/petsc4py via pip (either directly or during installation of tmmlib/tmm4py), the PETSc build system will look for any compilers in its path, starting with wrappers named mpicc,mpif90etc. If your system uses different wrappers (e.g., mpiifort instead of mpif90) or if you want to use specific compilers, you can specify them in the environment variable PETSC_CONFIGURE_OPTIONSbefore invoking pip install ... below. You can also specify optimization flags, paths to optimized BLAS/LAPACK libraries, and any other configuration options that PETSc accepts (see instructions for building PETSc from source below) For example:

export PETSC_CONFIGURE_OPTIONS='--with-cc=mpicc --with-fc=mpiifort --with-cxx=0 --with-blas-lapack-dir=$MKLROOT --COPTFLAGS="-O2" --FOPTFLAGS="-O2"'
pip install tmm4py

Or:

PETSC_CONFIGURE_OPTIONS='--with-cc=mpicc --with-fc=mpiifort --with-blas-lapack-dir=$MKLROOT --COPTFLAGS="-O2" --FOPTFLAGS="-O2"' pip install tmm4py

(1) If you don't have MPI or PETSc and only want to run models written in Python or use the C driver:

pip install mpich
pip install tmm4py

(2) You already have MPI but don't have PETSc:

Assuming the MPI compiler wrappers (mpicc, mpif90) are in your path, do:

pip install tmm4py

(3) You already have PETSc installed:

Set PETSC_DIR/PETSC_ARCH as per PETSc instructions. If you used pip to install PETSc then paths can be set like so:

export PETSC_DIR=`python3 -c "from petsc import get_petsc_dir; print(get_petsc_dir())"`
export PETSC_ARCH=''

And then:

pip install tmm4py

(4) You already have PETSc and petsc4py installed:

The previous instructions will install petsc4py. If you also already have petsc4py installed, either when building PETSc from source or via pip, you need to tell pip about it (see below) so that it won't build it again:

PETSC_HAS_PETSC4PY=1 pip install tmm4py

Notes:

(i) The reason pip needs to be told about an existing petsc4py is because it uses 'build isolation' when compiling packages. Each required dependency package is installed in an isolated environment that knows nothing about what you may already have installed in your system when you invoked 'pip install ...'. In fact, this process is recursive so that the dependencies of each dependency are similarly built in an isolated environment. To get around this and deal with all the possible permutations of how PETSc/petsc4py/TMM/tmm4py can be installed, a custom backend was created that dynamically figures out what the actual dependencies are and what needs to be installed based on your configuration. Thus, if the PETSC_DIR environment variable is detected everything will be built against that version.

(ii) petsc4py and tmm4py depend on NumPy at both build and run time. They also depend on Cython at build time. Ideally we want to use same versions of NumPy and Cython for petsc4py and tmm4py, as well as for building and at runtime. Unfortunately, this is difficult to enforce for petsc4py when using pip because its build system always downloads and builds against the latest compatible versions regardless of NumPy and Cython, regardless of what you already have installed in your Python environment (becaue of build isolation). This can be problematic on HPC systems with preinstalled versions of NumPy and other packages built against it. (That said, NumPy libraries at least are ABI-compatible across versions and unless you have very old versions of either NumPy and Cython none of this may pose an issue.) However, should you run into problems, you can ensure that petsc4py is built against your desired NumPy and Cython by turning off build isolation. First, install some required packages and (if they're not already installed) the NumPy and Cython versions you desire, for example:

pip install wheel setuptools numpy==1.24.4 cython==3.0.12

Then, install petsc4py without build isolation:

pip install petsc4py --no-build-isolation

In contrast, tmm4py's build system gives you much more control by letting you specify NumPy and Cython versions via environment variables, for example:

NUMPY_VER=1.24.4 C