starpy

Python scripts for easy RELION STAR file manipulation. Base library file metadata.py needed by all scripts.

!!! Scripts are now RELION 3.1+ compatible !!!

Now many of the scripts supports STDIN/STDOUT for input/output.

Backward compatible with RELION <=3.0 format star files!

Table of Contents

• General usage
• add_beamtiltclass_star.py
• add_remove_label.py
• analyze_orientation_distances_star.py
• assign_column_star.py
• binning_correct_star.py
• create_beamtiltclass_from_mdoc.py
• create_beamtiltclass_from_xml.py
• create_opticgroups_from_filename.py
• create_opticgroups_from_xml.py
• create_opticgroups_from_mdoc.py
• extract_particles_coords_star.py
• filter_astigmatism_star.py
• flip_particles_coordinates.py
• get_absolute_apix.py
• heatmap_orient_star.py
• helix_correct_star.py
• join_star.py
• json2star.py
• math_star.py
• math_exp_star.py
• metadata.py
• micrograph_star_from_particles_star.py
• particles_star_to_box.py
• particles_star_to_coords_star.py
• plot_star.py
• print_symmetry_matrices.py
• regular_box_pattern_around_center_coordinate.py
• rel31_to_rel30_star.py
• remove_preferred_orient.py
• remove_preferred_orient_hlpx.py
• rename_foilhole_star.py
• rotate_particles_star.py
• scf_star.py
• select_maxprob_sym_copy_ptcls.py
• select_orientations_star.py
• select_rand_sym_copy_ptcls.py
• select_values_star.py
• split_particles_to_micrographs.py
• split_stacks.py
• star_to_csv.py
• stats_star.py
• unbin_coordinates.py
• xflip_particles_star.py
• yflip_particles_star.py

Install

To install the scripts, clone the repository and add the path to the scripts to your PATH variable.

Some of the script need external libraries listed in requirements.txt. Install them ideally into separate virtual environment:

# Clone the repository
git clone https://github.com/fuzikt/starpy.git

# Change to the directory
cd starpy

# Create a virtual environment
python -m venv starpy_env

# Activate the virtual environment
source starpy_env/bin/activate

# Install requirements from the requirements.txt file
pip install -r requirements.txt

General usage

All scripts are designed to be used in the command line. To show the help of the script use the --h parameter.

The input and output files are defined by the --i and --o parameters.

In many scripts (see the description below) there is possibility to pass STDIN as input and STDOUT as output. This makes it possible to chain the result of one script to another using the pipe |.

If the --i parameter is not defined, the script will read from the standard input (STDIN). If the --o parameter is not defined, the script will write to the standard output (STDOUT).

Example 1: Plot the astigmatism calculated by math_exp_star.py of particles from input.star using plot_star.py

math_exp_star.py --i input.star --exp "abs(rlnDefocusU-rlnDefocusV)" --res_lb rlnResult | plot_star.py --lby rlnResult --hist_bin 20 --show

Example 2: Y-flip the particle from class2, then rotate them by r:15, t:20, p:150, remove preferred orientations and plot the orientations.

yflip_particles_star.py --i input.star --cls_nr 2 | rotate_particles_star.py --rot 15 --tilt 20 --psi 150 | remove_preferred_orient_hlpx.py | heatmap_orient_star.py --show

Example 3: Select particles from input.star where the difference between rlnAngleRot and rlnAngleTilt is less than 5 and store it in output.star.

math_exp_star.py --i input.star --exp  "1" --sel_exp "abs(rlnAngleRot-rlnAngleTilt)<5" --res_lb rlnResult  --def_val 0 | select_values_star.py --o output.star --lb rlnResult --op ">" --val 0

add_beamtiltclass_star.py

! only Relion <=3.0 format star files ! Add beamtilt class to the particles. Script adds rlnBeamTiltClass extracted from the micrograph name (in FoilHoleXXXX.mrc FEI format).

  --i    Input STAR filename (Default: STDIN).
  --o    Output STAR filename (Default: STDOUT).

add_remove_label.py

Adds or removes labels from star file.

--i         Input STAR filename (Default: STDIN).
--o         Output STAR filename (Default: STDOUT).
--add       Add new label to the star file.
--rm        Remove label from the star file.
--lb        Label to be added or removed. Use comma separated label values to add or remove multiple labels. Default: None
--val       Value filled for added labels. Use comma separated values if adding multiple labels. To copy values from existing label into the new one use the label name as value (e.g. rlnCoordinateX). Default: 0
--data      Data table from star file to be used (Default: data_particles).

Example 1: Remove rlnCoordinateY from input.star

add_remove_label.py --i input.star --o output.star --lb rlnCoordinateY --rm

Example 2: Remove rlnCoordinateX,rlnCoordinateY from input.star

add_remove_label.py --i input.star --o output.star --lb rlnCoordinateX,rlnCoordinateY --rm

Example 3: Add rlnCoordinateX,rlnCoordinateY with default values 10,20 respectively

add_remove_label.py --i input.star --o output.star --lb rlnCoordinateX,rlnCoordinateY --add --val 10,20

Example 4: Add rlnCoordinateZ and rlnAngleRot with values equal rlnCoordinateX and 20.2 respectively

add_remove_label.py --i input.star --o output.star --lb rlnCoordinateZ,rlnAngleRot --add --val rlnCoordinateX,20.2

analyze_orientation_distances_star.py

Calculates the spatial distance and angular distance between corresponding particles in --i1 and --i2. Output contains the particles from --i1 with additional columns for the spatial (rlnSpatDist), angular distances (rlnAngDist), rlnOriginXAngstDiff, rlnOriginYAngstDiff, rlnAngleRotDiff , rlnAngleTiltDiff, and rlnAnglePsiDiff.

  --i1    Input1 STAR filename (Default: STDIN).
  --i2    Input2 STAR filename (Default: STDIN).e
  --o     Output STAR filename (Default: STDOUT).

assign_column_star.py

Add label (col_lb) to Input1 and assigns values to it from Input2 where the label (comp_lb) of Input2 matches Input1

  --i1        Input1 STAR filename (Default: STDIN).
  --i2        Input2 STAR filename (Default: STDIN)..
  --o         Output STAR filename (Default: STDOUT).
  --data  Data table from star file to be used, Default: data_particles
  --col_lb    Label of the new column assigned to Input1; Default: rlnDefocusU
  --comp_lb   Compare label used for Input1 and Input2 for value assignment. Default:rlnMicrographName

Example 1: Assign values of DefocusU from input2.star as a column to input1.star where the value of column rlnMicrographName matches in both inputs.

assign_column_star.py --i1 input1.star --i2 input2.star --o output.star --col_lb rlnDefocusU --comp_lb rlnMicrographName

binnig_correct_star.py

Binning correct original star file according to the binning factor. Correcting rlnOriginX, rlnOriginY, pixel size, and particle suffix.

  --i             Input STAR filename with particles.
  --o             Output STAR filename.
  --bin_factor    Binning factor.
  --suf_orig      Original suffix to replace (e.g. _512.mrcs)
  --suf_new       New suffix to use for replacement (e.g. _256.mrcs)

create_beamtiltclass_from_mdoc.py

! only Relion <=3.0 format star files ! Clusters beam-shifts extracted from serialem mdoc files into beam-tilt classes.

  --i         Input mdoc directory
  --o         Output star file. If empty no file generated generated
  --o_shifts  Output file with extracted beam-shifts and cluster numbers. If empty no file generated generated
  --clusters  Number of clusters the beam-shifts should be divided in. (default: 1)
  --elbow     Number of max clusters used in Elbow method optimal cluster number determination. (default: 0)
  --max_iter  Expert option: Maximum number of iterations of the k-means algorithm for a single run. (default: 300)
  --n_init    Expert option: Number of time the k-means algorithm will be run with different centroid seeds. (default: 10)

Requires specific conda environment. Install:

conda create -n beamtiltclass-env
conda activate beamtiltclass-env
conda install scikit-learn
conda install matplotlib

create_beamtiltclass_from_xml.py

! only Relion <=3.0 format star files ! Clusters beam-shifts extracted from xml files into beam-tilt classes.

  --i         Input XML directory
  --o         Output star file. If empty no file generated generated
  --o_shifts  Output file with extracted beam-shifts and cluster numbers. If empty no file generated generated
  --clusters  Number of clusters the beam-shifts should be divided in. (default: 1)
  --elbow     Number of max clusters used in Elbow method optimal cluster number determination. (default: 0)
  --max_iter  Expert option: Maximum number of