reportIT

IonTorrent Variant Reporting Pipeline

This program will annotate, aggregate, and summarize clinical variant information from the IonTorrent suite

A brief graphical overview of the pipeline can be found here.

Overview

Sequencing results are accessed directly from the lab's IonTorrent server via scripts using ssh and rsync. After downloading files for a given run to the local system, VCF formatted variant call files are annotated and summarized to identify mutations of known significance (ANNOVAR, bcftools), while BAM formatted coverage files are visualized with IGV. An HTML formatted report is generated from variant summary information and IGV snapshots. Results can be easily emailed to clinicians for review with the provided script.

In Progress

Development of the following items is currently planned for the future:

Per-sample reports showing variant summary table and clinical interpretation of variants supplied by the Weill Cornell Precision Medicine Knowledgebase.
Analysis review feedback system to mark sequencing artifacts and remove them from report output
Deposition of pipeline output in a central database (REDCap, or other)

Analysis Report Example

[[ A full HTML version of the report can be previewed here or here. ]]

An analysis overview report displays the significant variants found across all samples in the run. 'SC' sensitivity control samples are shown in a separate table (hidden by default).

IGV snapshots shown for all significant variants. For low frequency variants, a "long view" snapshot is included to ensure mutations can be seen amongst reads. If available, 'NC' control sample is included on the lower track.

Installation

First, clone this repo:

git clone --recursive https://github.com/stevekm/reportIT.git
cd reportIT

Then, run the dir_setup.sh script;

./dir_setup.sh

This should set up the bin and ref directories, along with creating and symlinking the external input, output, and data directories. You should verify these symlinks and directories, and then populate the data directory with files required for the pipeline (see the 'Data directory' section, below). You should also set up automatic ssh for your IonTorrent server as described here.

Review the items described in the Files & Directories and Pipeline Settings sections to make sure everything is set properly.

Usage

Check For Runs

Before you can run the pipeline, you need to know which runs are available on your IonTorrent server. The following steps require that your data/server_info.txt file is set correctly, as described below. It is also recommended to have ssh key authentication set up for your user account on the IonTorrent server.

Missing Runs Only

If you only want to know which runs on the IonTorrent server are not present on your local system, you can use this script:

code/check_for_new_runs.py

By default, it will validate each missing run to make sure that IonTorrent sequencing data has been produced in the remote run directory. It will also automatically create an unpaired samplesheet for the missing runs. If you inlclude the -d argument to the script, it will also download the missing runs entered on the sample sheet produced.

All Available Runs

The following script will log into your IonTorrent server, and output a list of all run directories found:

code/get_server_run_list.sh

Make Samplesheet

The best way to run the reportIT pipeline is by using a samplesheet. These are stored in the samplesheets directory by default, and an example can be found here. A samplesheet must be in TSV (tab-separated) format, preferably with one run ID per line. If two runs should be treated as a 'pair', then both run ID's should be on the same line. Note that paired run processing only affects report and IGV snapshot generation, not downloading or annotation.

The best way to create a samplesheet is to use the make_samplesheet.py script. This script can take any number of unpaired run ID's, and a single set of paired ID's.

code/make_samplesheet.py unpaired_run1 unpaired_run2 -p paired_run3.1 -p paired_run3.2

A samplesheet produced this way will look like this:

unpaired_run1
unpaired_run2
paired_run3.1	paired_run3.2

Run Pipeline

The simplest way to run the reportIT pipeline is to use the run_samplesheet.py script, and specify which actions you would like to take on the analysis ID's specified in your sample sheet. The following actions are available:

Download: -d
Annotate: -a
Report: -r

The following modifiers are available:

Submit to cluster with qsub: -q

Multiple actions can be combined in a single command:

# download all files, then annotate and report with qsub
$ code/run_samplesheet.py samplesheets/samplesheet.tsv -darq

NOTE: The -q method has been configured to work with the phoenix SGE compute cluster at NYULMC, and might need to be reconfigured to work on other HPC systems.

Mail Results

After manually reviewing the HTML report output, pipeline results can be delivered in a pre-formatted email using the following script:

code/mail_analysis_report.sh <analysis_ID>

Multiple analysis_ID's can be passed to email all results sequentially. All summary tables, VCF files, IGV snapshots, and the analysis overview report will be included as email attachments. Configuration for emailing is saved in the file mail_settings.sh.

Usage Notes

Rough estimates for pipeline completeion time are ~5-10 minutes to download all files and annotate variants, and ~5-15 minutes to create all IGV snapshots and generate reports. In total this comes to roughly 10 - 30 minutes per analysis, depending on the number of variants present.

Running the pipeline without the -q argument will run all pipeline steps for all analyses in the current session; if you plan to do this, you should probably run the pipeline in screen in order to allow it to run in the background indepedent of your terminal connection. Note that running with qsub is currently disabled for the file download step, so all file downloads will always run in the current session. If you have a lot of analyses, this might take a while.

Program Validation

As a safety feature against undesired usage, the run_samplesheet.py script includes self-validating features to make sure the following items are set correctly before running the pipeline:

check that the proper git branch is currently in use
check that the proper output directory has been symlinked

These validations can be skipped by adding the --debug argument to the script.

Files & Directories

Input, output, and reference data for the program is stored external to the program's directory and is set by symlinks. These should be automatically created when you run the dir_setup.sh script during initial installation.

Data directory

The data directory should contain the following items:

data/
|-- control_sample_IDs.txt
|-- control_sample_regex.txt
|-- email_recipients.txt
|-- actionable_genes.txt
|-- panel_genes.txt
|-- server_info.txt
`-- summary_fields.txt

Important files:

control_sample_IDs.txt: ID's for IonTorrent samples which are used to denote 'control' samples, which should not be used for IGV snapshots. Example:

NC
SC
NTC
NC HAPMAP
HAPMAP
Sc
SC-ACROMETRIX

control_sample_regex.txt: Regex patterns to use with grep -F in some scripts which try to identify NC control samples. Example:

^NC[[:space:]]
^NC[[:space:]]HAPMAP[[:space:]]
^HAPMAP[[:space:]]

SC_control_sample_IDs.txt: ID's to identify SC control samples. Example:

SC
Sc
SC-ACROMETRIX

email_recipients.txt: A list of email addresses to use in the To: field of the outgoing email with analysis results. The addresses must be on a single line, formatted as such:

email1@gmail.com, email2@gmail.com

panel_genes.txt : A list of genes to be included in the gene panel. Example:

AKT1
ALK
APC
ATM

actionable_genes.txt : A list of genes determined to be actionable. Example:

BRAF
EGFR
FLT3

server_info.txt : The login info for the IonTorrent server. Must be formatted as such:

username@server_IP

References Directory

This directory contains the following items:

ref
|-- cannonical_transcript_table.py
`-- hg19
    |-- canonical_transcript_list.txt
    |-- download_refs.txt
    |-- kgXref.txt
    |-- kgXref.txt.gz
    |-- knownCanonical.txt
    `-- knownCanonical.txt.gz