DAPPER

<!-- ! ___ _ ___ ___ ___ ___ ! | \ /_\ | _ \ _ \ __| _ \ ! | |) / _ \| _/ _/ _|| / ! |___/_/ \_\_| |_| |___|_|_\ ! ! --> <img src="https://github.com/nansencenter/DAPPER/raw/master/docs/images/logos/logo_wtxt.png" alt="DAPPER logo" align="left" width="250"/>

DAPPER is a set of templates for benchmarking the performance of data assimilation (DA) methods. The numerical experiments provide support and guidance for new developments in DA. The typical set-up is a synthetic (twin) experiment, where you specify a dynamic model and an observational model, and use these to generate a synthetic truth (multivariate time series), and then estimate that truth given the models and noisy observations.

<!-- Badges / shields -->

Getting started

• Read & run examples/basic_1.py and basic_2.py, or their corresponding notebooks (requires Google login).
• This screencast provides an overview to DAPPER.
• Install.
• The documentation includes general guidelines and the API reference, but most users must expect to read the code as well.
• If used towards a publication, please cite as The experiments used (inspiration from) DAPPER [ref], version 1.6.0, or similar, where [ref] points to .
• Also see the interactive tutorials on DA theory with Python.

Highlights

DAPPER enables the numerical investigation of DA methods through a variety of typical test cases and statistics. It (a) reproduces numerical benchmarks results reported in the literature, and (b) facilitates comparative studies, thus promoting the (a) reliability and (b) relevance of the results. For example, the figure below is generated by docs/examples/basic_3.py, reproduces figure 5.7 of these lecture notes. DAPPER is (c) open source, written in Python, and (d) focuses on readability; this promotes the (c) reproduction and (d) dissemination of the underlying science, and makes it easy to adapt and extend.

DAPPER demonstrates how to parallelise ensemble forecasts (e.g., the QG model), local analyses (e.g., the LETKF), and independent experiments (e.g., docs/examples/basic_3.py). It includes a battery of diagnostics and statistics, which all get averaged over subdomains (e.g., "ocean" and "land") and then in time. Confidence intervals are computed, including correction for auto-correlations, and used for uncertainty quantification, and significant digits printing. Several diagnostics are included in the on-line "liveplotting" illustrated below, which may be paused for further interactive inspection. In summary, DAPPER is well suited for teaching and fundamental DA research. Also see its drawbacks.

<!-- Non-highlighted features: - Time sequences use via `tools.chronos.Chronology` and `tools.chronos.Ticker`. - Random variables via `tools.randvars.RV`: Gaussian, Student-t, Laplace, Uniform, ..., as well as support for custom sampling functions. - Covariance matrices via `tools.matrices.CovMat`: provides input flexibility/overloading, lazy eval that facilitates the use of non-diagonal covariance matrices (whether sparse or full). - built-in tools for experiment and result management, -->

Installation

Successfully tested on Linux/Mac/Windows.

Prerequisite: Python>=3.12

If you're an expert, setup a python environment however you like. Otherwise: Install Anaconda, then open the Anaconda terminal and run the following commands:

conda create --yes --name dapper-env python=3.12
conda activate dapper-env
python --version

Ensure the printed version is as desired. Keep using the same terminal for the commands below.

Install

Either: Install for development (recommended)

Do you want the DAPPER code available to play around with? Then

• Download and unzip (or git clone) DAPPER.
• Move the resulting folder wherever you like,
  and cd into it (ensure you're in the folder with a setup.py file).
• pip install -e '.'

Or: Install as library

Do you just want to run a script that requires DAPPER? Then

• If the script comes with a requirements.txt file that lists DAPPER, then do
  pip install -r path/to/requirements.txt.
• If not, hopefully you know the version of DAPPER needed. Run
  pip install dapper==1.6.0 to get version 1.6.0 (as an example).

Finally: Test the installation

You should now be able to do run your script with python path/to/script.py.
For example, if you are in the DAPPER dir,

python docs/examples/basic_1.py

PS: If you closed the terminal (or shut down your computer), you'll first need to run conda activate dapper-env

DA methods

<!-- markdownlint-capture --> <!-- markdownlint-disable line-length -->

| Method | Literature reproduced | |------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------| | EnKF ¹ | Sakov08, Hoteit15, Grudzien2020 | | EnKF-N | Bocquet12, Bocquet15 | | EnKS, EnRTS | Raanes2016 | | iEnKS / iEnKF / EnRML / ES-MDA ² | Sakov12, Bocquet12, Bocquet14 | | LETKF, local & serial EAKF | Bocquet11 | | Sqrt. model noise methods | Raanes2014 | | Particle filter (bootstrap) ³ | Bocquet10 | | Optimal/implicit Particle filter ³ | Bocquet10 | | NETF | Tödter15, Wiljes16 | | Rank histogram filter (RHF) | Anderson10 | | 4D-Var | | | 3D-Var | | | Extended KF | | | Optimal interpolation | | | Climatology |