FastLZ

Overview

FastLZ (MIT license) is an ANSI C/C90 implementation of Lempel-Ziv 77 algorithm (LZ77) of lossless data compression. It is suitable to compress series of text/paragraphs, sequences of raw pixel data, or any other blocks of data with lots of repetition. It is not intended to be used on images, videos, and other formats of data typically already in an optimal compressed form.

The focus for FastLZ is a very fast compression and decompression, doing that at the cost of the compression ratio. As an illustration, the comparison with zlib when compressing enwik8 (also in more details):

||Ratio|Compression|Decompression |--|--|--|--| |FastLZ |54.2%|159 MB/s|305 MB/s| |zlib -1|42.3%|50 MB/s|184 MB/s| |zlib -9|36.5%|11 MB/s|185 MB/s|

FastLZ is used by many software products, from a number of games (such as Death Stranding) to various open-source projects (Godot Engine, Facebook HHVM, Apache Traffic Server, Calligra Office, OSv, Netty, etc). It even serves as the basis for other compression projects like BLOSC.

For other implementations of byte-aligned LZ77, take a look at LZ4, Snappy, Density, LZO, LZF, LZJB, LZRW, etc.

Usage

FastLZ can be used directly in any C/C++ applications. For other programming languages/environments, use the corresponding binding:

• Rust, available on Crates: cargo install fastlz
• Python, available on PyPi: pip install fastlz
• JavaScript, available on npm: npm install fastlz
• Ruby, available on Rubygems: gem install fastlz
• Lua via github.com/oneoo/lua-fastlz

FastLZ consists of only two files: fastlz.h and fastlz.c. Just add these files to your project in order to use FastLZ. For the detailed information on the API to perform compression and decompression, see fastlz.h.

For Vcpkg users, FastLZ is already available: vcpkg install fastlz.

A simple file compressor called 6pack is included as an example on how to use FastLZ. The corresponding decompressor is 6unpack.

FastLZ supports any standard-conforming ANSI C/C90 compiler, including the popular ones such as GCC, Clang, Visual Studio, and even Tiny CC. FastLZ works well on a number of architectures (32-bit and 64-bit, big endian and little endian), from Intel/AMD, PowerPC, System z, ARM, MIPS, and RISC-V.

The continuous integration system runs an extensive set of compression-decompression round trips on the following systems:

For more details, check the corresponding GitHub Actions build logs.

| | | | | |----------------------|--------------------------------------------------------------------------------------------------------:|--------------------------------------------------------------------------------------------------:|--------------------------------------------------------------------------------------------:| | amd64 | Linux | Windows | macOS | | GCC | | | | | Clang | | | | | TinyCC | | | | | VS 2019 | | | | | i686 | Linux | Windows | macOS | | GCC | | | | | Clang | | | | | TinyCC | | | | | VS 2019 | | | | | i586 | Linux | DOS | | | GCC | | | | | | Linux | | | | powerpc | | | | | GCC | | | | | ppc64(le) | | | | | GCC | | |