pyGadgetReader

Author: Robert Thompson

E-Mail: rthompsonj@gmail.com

If you use this code for your work please cite [ascl:1411.001]

Summary
Requirements
Obtaining and Updating
Customization
Installation
Usage
Tips

Summary

Do you love running simulations but hate fighting with the different flavors of output? If so, you've come to the right place! pyGadgetReader is designed to take the headache out of reading your GADGET simulation data; it plays the role of interpreter between the binary snapshot & python. The module currently supports the following data types:

Gadget types 1 & 2 binaries (single/multi-part)
HDF5 outputs (single/multi-part)
TIPSY binaries (bin/aux)

pyGadgetReader attempts to detect which file format is is dealing with so you don't have to! It does however, assume that the different formats have unique file extensions (none,.hdf5,& .bin).

pyGadgetReader also supports the following non-GADGET files:

Rockstar binary outputs
Rockstar-Galaxies binary outputs
FoF Special catalogue & indexlist binaries
P-StarGroupFinder data files

Requirements

python >= 2.7.x
numpy >= 1.7.x
h5py

Obtaining and Updating

The easiest way to download the code and stay up to date is to clone a version from bitbucket to your local computer via Mercurial (hg):

> hg clone https://bitbucket.org/rthompson/pygadgetreader

To update all you need to do is:

> hg pull
> hg up
(reinstall)

Customization

Before building the module, there are a few customizations you may want to tinker with. The first two variables can be found in readgadget/modules/common.py, while the third is located in readgadget/modules/gadget_blockordering.py.

UNITS: The code currently assumes that your default Gadget length units are kpc/h, mass units are 10^(10)Msun/h, and velocity units are km/s. This can be changed by modifying the UnitLength_in_cm, UnitMass_in_g, and UnitVelocity_in_cm_per_s respectively in readgadget/modules/common.py. This can optionally be altered on a per-call basis by passing new values through as an option to the readsnap() function. (note: has NO impact on TIPSY files as of now)
METALFACTOR: If your metal field contains multiple species (flag_metals > 1), this factor is multiplied to their sum to determine the particle's overall metallicity (default is 0.0189/0.0147 for historical reasons).
BLOCKORDERING: When dealing with Gadget type-1 binaries, the block-ordering can be a pain in the neck. Custom fields can be added the snapshot causing all of your previous readers to start returning bad data; not anymore! I've tried to design a system where the user can easily customize their block ordering. pyGadgetReader currently has 2 default options - BLOCKORDERING0 and BLOCKORDERING1. The default is set via the DEFAULT_BLOCKORDERING variable, which is used in conjunction with the BLOCKORDERING dictionary defined farther down in the file. This can be changed by editing readgadget/modules/gadget_blockordering.py.

If you require a custom block ordering, use one of the already present block orderings as a template. The first step is creating a new OrderedDict named something along the lines of BLOCKORDERING3. Each entry in the OrderedDict represents a data block, and must contain both a KEY and a VALUE. Once your new block ordering is defined, you should add it to the BLOCKORDERING dictionary with an appropriate name. You can also set it as default via the DEFAULT_BLOCKORDERING variable if you so wish.

KEY: This is the name of the block - it MUST be present as a value in the dataTypes dictionary defined in readgadget/modules/names.py.

VALUE: This is a list that contains 2 important pieces of information: 1) what particles this block is present for, and 2) should the code first check for a specific header flag?

Below are two examples. The first represents the 'pos' block (which is present in the dataTypes dictionary). The second entry is a list telling the code what particle types have this block, where -1 meaning ALL particle types. The second represents the 'metallicity' data block; here we have a list of [0,4] telling the code that this block is present for particle types 0 (gas) & 4 (stars), and to check the flag_metals flag before attempting a read. You can omit the flag checker if you know for certain the data block exists.

('pos',[-1]),
('metallicity',[[0,4],'flag_metals']),

Installation

Once the code is downloaded there are two methods of installation depending on your access rights. If you have write access to your python distribution, then the preferred method is to execute the following commands:

	> python setup.py build     ## this builds the module
	> python setup.py install   ## this installs the module, may require sudo

If you do not have write access to your python install, we need to modify your environment variable PYTHONPATH to point to the pyGadgetReader directory. Add these two lines to your .bashrc/.bash_profile (or respective shell file):

PYTHONPATH=/path/to/pygadgetreader:$PYTHONPATH
export PYTHONPATH

NOTE for uninstalling previous versions:

If you had previously installed my C version of pyGadgetReader you should remove it before trying to use the code as there may be some naming conflicts. First you need to find out where python, a point in the general direction is typing which python in your terminal, this will return your installation directory. Next you need to locate your site-packages directory which is usually under python's lib directory. Once there you are looking for anything in the form of readgadget.so, once this is found remove it.

Usage

IMPORTANT: When using pyGadgetReader, try NOT to include the snapshot extension or number prefix (for multi-part). As an example, if your snapshot is named 'snap_N128L16_005.0.hdf5', you would only pass 'snap_N128L16_005' to the below functions. If the code detects .hdf5, .bin, or .0 in the snapname it will attempt to strip the extension.

To gain access to the following functions, place this at the top of your python script:

from pygadgetreader import *

readheader()

This function reads in the header and returns values of interest. The values it can read in are as follows:

 time	       - scale factor of the snapshot
 redshift      - redshift of the snapshot
 boxsize       - boxsize if present (typically in kpc/h)
 O0	       	   - Omega_0 (Omega_dm+Omega_b)
 Ol	       	   - Omega_Lambda
 h	       	   - hubble parameter
 gascount      - total # of gas   particles [type 0]
 dmcount       - total # of DM    particles [type 1]
 diskcount     - total # of disk  particles [type 2]
 bulgecount    - total # of bulge particles [type 3]
 starcount     - total # of star  particles [type 4]
 bndrycount    - total # of bndry particles [type 5]
 f_sfr	       - Star Formation Rate flag	 0=off 1=on
 f_fb	       - Feedback flag	     		 0=off 1=on
 f_cooling     - Cooling flag			 	 0=off 1=on 
 f_age	       - Stellar Age tracking flag	 0=off 1=on
 f_metals      - Metal tracking flag  		 0=off 1=on

 npartTotal    - list of particle counts (including HighWord)
 header		   - returns the entire header as a dictionary
 
 npartThisFile - list of particle counts in a single file
                 (you MUST pass the full snapshot name for this one, 
                  otherwise it will return data from file .0)

DEFINITION:

	readheader(a,b,debug=0,single=0)

	Parameters
   	----------
   	a : string
   	    Input file
	b : string
	    Requested data type (from above list)

	Optional
	--------
	debug: output debugging info

	Returns
	-------
	float, double, int, list, array, dict (depending on request)

	Examples
	--------
	>>> readheader('snap_001','redshift')
		2.3499995966280447

	>>> readheader('snap_005','npartTotal')
		array([  53921,   54480,       0,       0,    3510, 2090342], dtype=uint32)
	
	>>> readheader('snap_006','header')
		{'O0': 0.29999999999999999,
		 'Ol': 0.69999999999999996,
		 'boxsize': 16000.0,
		 'h': 0.69999999999999996,
		 'massTable': array([ 0.,  0.00172772,  0.,  0.,  0., 0.01626092]),
		  ...,
		 'time': 0.29850749862971743}

readsnap()

This function does the heavy lifting. It reads data blocks from the snapshot and returns the requested data for a a specified particle type.

SUPPORTED DATA BLOCKS:

  ---------------------
  -  STANDARD BLOCKS  -
  ---------------------
   pos	       - (all)         Position data
   vel	       - (all)         Velocity data code units
   pid	       - (all)         Particle ids
   mass	       - (all)         Particle masses
   u	       - (gas)         Internal energy
   rho	       - (gas)         Density
   ne	       - (gas)         Number density of free electrons
   nh	       - (gas)         Number densi

Pygadgetreader

Install / Use

README