VSEARCH

Introduction

The aim of this project is to create an alternative to the USEARCH tool developed by Robert C. Edgar (2010). The new tool should:

• have open source code with an appropriate open source license
• be free of charge, gratis
• have a 64-bit design that handles very large databases and much more than 4GB of memory
• be as accurate or more accurate than usearch
• be as fast or faster than usearch

We have implemented a tool called VSEARCH which supports de novo and reference based chimera detection, clustering, full-length and prefix dereplication, rereplication, reverse complementation, masking, all-vs-all pairwise global alignment, exact and global alignment searching, shuffling, subsampling and sorting. It also supports FASTQ file analysis, filtering, conversion and merging of paired-end reads.

VSEARCH stands for vectorized search, as the tool takes advantage of parallelism in the form of SIMD vectorization as well as multiple threads to perform accurate alignments at high speed. VSEARCH uses an optimal global aligner (full dynamic programming Needleman-Wunsch), in contrast to USEARCH which by default uses a heuristic seed and extend aligner. This usually results in more accurate alignments and overall improved sensitivity (recall) with VSEARCH, especially for alignments with gaps.

VSEARCH binaries are provided for GNU/Linux on five 64-bit processor architectures: x86_64, POWER8 (ppc64le), ARMv8 (aarch64), little-endian 64-bit RISC-V (riscv64), and little-endian 64-bit MIPS (mips64el). Binaries are also provided for macOS (version 10.9 Mavericks or later) on Intel (x86_64) and Apple Silicon (ARMv8), as well as Windows (64-bit, version 7 or higher, on x86_64). VSEARCH contains native SIMD code for three processor architectures (SSE2/SSSE3, AltiVec/VMX/VSX, Neon). In addition, VSEARCH uses the SIMD Everywhere (SIMDe) library to enable building on riscv64, mips64el, and other little-endian architectures, but the performance may be lower than a native implementation.

| CPU \ OS | GNU/Linux | macOS | Windows | | ------------- | :-----------: | :----: | :-------: | | x86_64 | ✔ | ✔ | ✔ | | ARMv8 | ✔ | ✔ | | | POWER8 | ✔ | | | | RISC-V 64 LE | ✔ | | | | MIPS 64 LE | not tested | | |

Various packages, plugins and wrappers for VSEARCH are also available from other sources - see below.

The source code compiles correctly with gcc (versions 4.8.5 to 14.0) and llvm-clang (3.8 to 19.0). The source code should also compile on FreeBSD and NetBSD systems.

VSEARCH can directly read input query and database files that are compressed using gzip (.gz) and bzip2 (.bz2) if the zlib and bzip2 libraries are available.

Most of the nucleotide based commands and options in USEARCH version 7 are supported, as well as some in version 8. The same option names as in USEARCH version 7 has been used in order to make VSEARCH an almost drop-in replacement. VSEARCH does not support amino acid sequences or local alignments. These features may be added in the future.

Getting Help

If you can't find an answer in online documentation, or in the manpage, please visit the VSEARCH Web Forum to post a question or start a discussion.

Example

In the example below, VSEARCH will identify sequences in the file database.fsa that are at least 90% identical on the plus strand to the query sequences in the file queries.fsa and write the results to the file alnout.txt.

./vsearch --usearch_global queries.fsa --db database.fsa --id 0.9 --alnout alnout.txt

Download and install

Source distribution To download the source distribution from a release and build the executable and the documentation, use the following commands:

wget https://github.com/torognes/vsearch/archive/v2.30.6.tar.gz
tar xzf v2.30.6.tar.gz
cd vsearch-2.30.6
./autogen.sh
./configure CFLAGS="-O2" CXXFLAGS="-O2"
make ARFLAGS="cr"
sudo make install

You may customize the installation directory using the --prefix=DIR option to configure. If the compression libraries zlib and/or bzip2 are installed on the system, they will be detected automatically and support for compressed files will be included in vsearch (see section Dependencies below). Support for compressed files may be disabled using the --disable-zlib and --disable-bzip2 options to configure. A PDF version of the manual will be created from the vsearch.1 manual file if ps2pdf is available, unless disabled using the --disable-pdfman option to configure. It is recommended to run configure with the options CFLAGS="-O2" and CXXFLAGS="-O2". Other options may also be applied to configure, please run configure -h to see them all. GNU autoconf (version 2.63 or later), automake and the GCC C++ (g++) compiler is required to build vsearch. Version 3.82 or later of make may be required on Linux, while version 3.81 is sufficient on macOS.

Warning: Compiling the align_simd.cc file on x86_64 systems using the GNU C++ compiler version 9 or later with the -O3 optimization option on may result in incorrect code that may cause bad alignments in some circumstances. This was due to the -ftree-partial-pre optimization enabled by -O3. A compiler pragma has been inserted in the code to specifically turn off this optimization for the affected code. Using -O3 should be safe.

To build VSEARCH on Debian and similar Linux distributions (Ubuntu etc) you'll need the following packages: autoconf, automake, g++, ghostscript, groff, libbz2-dev, make, zlib1g-dev. Include libsimde-dev to build on riscv64 or mips64el.

To build VSEARCH on Fedora and similar Linux distributions (RHEL, Centos etc) you'll need the following packages: autoconf, automake, bzip2-devel, gcc-c++, ghostscript, groff-base, make, zlib-devel.

Instead of downloading the source distribution as a compressed archive, you could clone the repo and build it as shown below. The options to configure as described above are still valid.

git clone https://github.com/torognes/vsearch.git
cd vsearch
./autogen.sh
./configure CFLAGS="-O2" CXXFLAGS="-O2"
make ARFLAGS="cr"
sudo make install

Binary distribution: Starting with version 1.4.0, binary distribution files containing pre-compiled binaries as well as the documentation will be made available as part of each release. The included executables include support for input files compressed by zlib and bzip2 (with files usually ending in .gz or .bz2).

Binary distributions are provided for x86-64 systems running GNU/Linux, macOS (version 10.7 or higher) or Windows (64-bit, version 7 or higher), 64-bit AMDv8 (aarch64) systems running GNU/Linux or macOS, as well as POWER8 (ppc64le), 64-bit little-endian RISC-V (risv64), and 64-bit little endian MIPS (mips64el) systems running GNU/Linux. A universal macOS binary is also provided. In addition, an x86_64 binary built for the discontinued RHEL 7 and CentOS 7 linux distributions is provided. The other Linux binaries are built on Debian 11 (oldstable, Bullseye). Static binaries are available for all Linux architectures except x86_64, these can be used on systems that do not have all the necessary libraries installed. The Windows binary was built with cross compilation using Mingw-w64.

Download the appropriate executable for your system using the following commands if you are using a Linux or macOS system:

wget https://github.com/torognes/vsearch/releases/download/v{VERSION}/vsearch-{VERSION}-{OS}-{ARCH}.tar.gz
tar xzf vsearch-{VERSION}-{OS}-{ARCH}.tar.gz

Replace {VERSION} with the VSEARCH version number (e.g. 2.30.6), {OS} with the target operating system (linux or macos), and {ARCH} with the architecture (x86_64, aarch64, ppc64le, riscv64, or mips64el). You could add -static after {ARCH} to get a statically compiled version for Linux (except x86_64). The name of the binary for the RHEL 7 and CentOS 7 Linux distributions ends in -ubi7.

Or, if you are using Windows, download and extract (unzip) the contents of this file:

https://github.com/torognes/vsearch/releases/download/v{VERSION}/vsearch-{VERSION}-win-x86_64.zip

Linux and Mac: You will now have the binary distribution in a folder called vsearch-{VERSION}-{OS}-{ARCH} in which you will find three subfolders bin, man and doc. We recommend making a copy or a symbolic link to the vsearch binary bin/vsearch in a folder included in your $PATH, and a copy or a symbolic link to the vsearch man page man/vsearch.1 in a folder included in your $MANPATH. The PDF version of the manual is available in doc/vsearch_manual.pdf.

Windows: You will now have the binary distribution in a folder called vsearch-{VERSION}-win-x86_64. The vsearch executable is called vsearch.exe. The manual in PDF format is called vsearch_manual.pdf. If you want to be able to call vsearch.exe from any command prompt window, you can put the VSEARCH executable in a folder (for instance C:\Users\<yourname>\bin), and add the new folder to the user Path: open the Environment Variables window by searc