TreeCorr

.. image:: https://github.com/rmjarvis/TreeCorr/actions/workflows/ci.yml/badge.svg :target: https://github.com/rmjarvis/TreeCorr/actions/workflows/ci.yml .. image:: https://codecov.io/gh/rmjarvis/TreeCorr/branch/main/graph/badge.svg :target: https://codecov.io/gh/rmjarvis/TreeCorr

TreeCorr is a package for efficiently computing 2-point and 3-point correlation functions.

• The code is hosted at https://github.com/rmjarvis/TreeCorr
• It can compute correlations of regular number counts, weak lensing shears, or scalar quantities such as convergence or CMB temperature fluctuations.
• 2-point correlations may be auto-correlations or cross-correlations. This includes shear-shear (aka GG), count-shear (NG), count-count (NN), kappa-kappa (KK), etc. (Any combination of shear, kappa, and counts.)
• 3-point correlations include both auto-correlations (e.g. NNN, KKK, GGG) and mixed cross-correlations (e.g. NNG, NKK, KGG and permutations).
• Additional field types with non-zero spin are supported in two-point correlations, including vectors (spin-1, V), shear (spin-2, G), trefoil (spin-3, T), and quatrefoil (spin-4, Q). Three-point correlations currently support N, K, and G fields (including mixed combinations such as NNG, NKK, KGG and permutations).
• Both 2- and 3-point functions can be done with the correct curved-sky calculation using RA, Dec coordinates, on a Euclidean tangent plane, or in 3D using either (RA,Dec,r) or (x,y,z) positions.
• The front end is in Python, which can be used as a Python module or as a standalone executable using configuration files. (The executable is corr2 for 2-point and corr3 for 3-point.)
• The actual computation of the correlation functions is done in C++ using ball trees (similar to kd trees), which make the calculation extremely efficient.
• When available, OpenMP is used to run in parallel on multi-core machines.
• Approximate running time for 2-point shear-shear is ~30 sec * (N/10^6) / core for a bin size b=0.1 in log(r). It scales as b^(-2). This is the slowest of the various kinds of 2-point correlations, so others will be a bit faster, but with the same scaling with N and b.
• The running time for 3-point functions is highly variable depending on the range of triangle geometries you are calculating. They are significantly slower than the 2-point functions, but many orders of magnitude faster than brute force algorithms.
• If you use TreeCorr in published research, please reference: Jarvis, Bernstein, & Jain, 2004, MNRAS, 352, 338 (I'm working on new paper about TreeCorr, including some of the improvements I've made since then, but this will suffice as a reference for now.)
• If you use the three-point multipole functionality of TreeCorr, please also reference Porth et al, 2023, arXiv:2309.08601
• Record on the Astrophysics Source Code Library: https://ascl.net/1508.007
• Developed by Mike Jarvis. Feel free to contact me with questions or comments at mikejarvis17 at gmail. Or post an issue (see below) if you have any problems with the code.

The code is licensed under a FreeBSD license. Essentially, you can use the code in any way you want, but if you distribute it, you need to include the file TreeCorr_LICENSE with the distribution. See that file for details.

Installation

The easiest ways to install TreeCorr are either with pip::

pip install treecorr

or with conda::

conda install -c conda-forge treecorr

If you have previously installed TreeCorr, and want to upgrade to a new released version, you should do::

pip install treecorr --upgrade

or::

conda update -c conda-forge treecorr

Depending on the write permissions of the Python distribution for your specific system, you might need to use one of the following variants for pip installation::

sudo pip install treecorr
pip install treecorr --user

The latter installs the Python module into ~/.local/lib/python3.X/site-packages, which is normally already in your PYTHONPATH, but it puts the executables corr2 and corr3 into ~/.local/bin which is probably not in your PATH. To use these scripts, you should add this directory to your PATH. If you would rather install into a different prefix rather than ~/.local, you can use::

pip install treecorr --install-option="--prefix=PREFIX"

This would install the executables into PREFIX/bin and the Python module into PREFIX/lib/python3.X/site-packages.

If you would rather download the tarball and install TreeCorr yourself, that is also relatively straightforward:

1. Download TreeCorr ^^^^^^^^^^^^^^^^^^^^

   You can download the latest tarball from::

    https://github.com/rmjarvis/TreeCorr/releases/
   

   Or you can clone the repository using either of the following::

    git clone git@github.com:rmjarvis/TreeCorr.git
    git clone https://github.com/rmjarvis/TreeCorr.git
   

   which will start out in the current stable release branch.

   Either way, cd into the TreeCorr directory.

2. Install dependencies ^^^^^^^^^^^^^^^^^^^^^^^

   All required dependencies should be installed automatically for you by pip or conda, so you should not need to worry about these. But if you are interested, the dependencies are:

   • numpy
   • pyyaml
   • LSSTDESC.Coord
   • pybind11

   They can all be installed at once by running::

    pip install -r requirements.txt
   

   or::

    conda install -c conda-forge treecorr --only-deps
   

   .. note::

    Several additional modules are not required for basic TreeCorr operations, but are
    potentially useful.
   
    - fitsio is required for reading FITS catalogs or writing to FITS output files.
    - pandas will significantly speed up reading from ASCII catalogs.
    - pandas and pyarrow are required for reading Parquet files.
    - h5py is required for reading HDF5 catalogs.
    - mpi4py is required for running TreeCorr across multiple machines using MPI.
   
    These are all pip installable::
   
        pip install fitsio
        pip install pandas
        pip install pyarrow
        pip install h5py
        pip install mpi4py
   
    But they are not installed with TreeCorr automatically.
   
    Also, beware that many HPC machines (e.g. NERSC) have special instructions for installing
    mpi4py, so the above command may not work on such systems.  If you have trouble, look
    for specialized instructions about how to install mpi4py properly on your system.
   

3. Install ^^^^^^^^^^

   You can then install TreeCorr from the local distribution. Typically this would be::

    pip install .
   

   If you don't have write permission in your Python distribution, you might need to use::

    pip install . --user
   

   In addition to installing the Python module treecorr, this will install the executables corr2 and corr3 in a bin folder somewhere on your system. Look for a line like::

    Installing corr2 script to /anaconda3/bin
   

   or similar in the output to see where the scripts are installed. If the directory is not in your PATH, you will also get a warning message at the end letting you know which directory you should add to your path if you want to run these scripts.

4. Run Tests (optional) ^^^^^^^^^^^^^^^^^^^^^^^

   If you want to run the unit tests, you can do the following::

    pip install -r test_requirements.txt
    cd tests
    pytest
   

Two-point Correlations

TreeCorr can compute a variety of two-point correlations:

:NN: The normal two-point correlation function of number counts (typically galaxy counts).

:GG: Two-point shear-shear correlation function.

:KK: Nominally the two-point kappa-kappa correlation function, although any scalar quantity can be used as "kappa". In lensing, kappa is the convergence, but this could be used for temperature, size, etc.

:NG: Cross-correlation of counts with shear. This is what is often called galaxy-galaxy lensing.

:NK: Cross-correlation of counts with kappa. Again, "kappa" here can be any scalar quantity.

:KG: Cross-correlation of convergence with shear. Like the NG calculation, but weighting the pairs by the kappa values of the foreground points.

There are also additional combinations involving complex fields with different spin than 2 (shear is a spin-2 field). See Two-point Correlation Functions <https://rmjarvis.github.io/TreeCorr/_build/html/correlation2.html>_ for more details.

Three-point Correlations

Three-point correlation functions are significantly more complicated, being functions of three parameters defining the triangle size and shape, rather than just a single separation. For cross-correlations, there are also issues related to whether one wants to allow the different catalogs to take all possible vertices in the triangles or be fixed to a particular vertex.

TreeCorr can compute the following three-point auto-correlations:

:NNN: Three-point correlation function of number counts.

:GGG: Three-point shear correlation function. We use the "natural components" called Gamma, described by Schneider & Lombardi (2003) (Astron.Astrophys. 397, 809) using the triangle centroid as the reference point.

:KKK: Three-point kappa correlation function. Again, "kappa" here can be any scalar quantity.

It is also possible to compute cross-correlations combining two of these types, such as NNG, NKK, KGK, etc. The ordering of the letters indicates which type is placed at each numbered vertex in the triangle: the first vertex is opposite d1, the second is opposite d2, and the third is opposite d3. The meaning of the three side lengths is particular to the choice of binning.

See Three-point Correlation Functions <https://rmjarvis.github.io/TreeCorr/_build/html/correlation3.html>_ for more