blz4

About

This is an example using some of the compression algorithms from BriefLZ to produce output in the format of LZ4.

Please note: this is just a quick experiment to see how it would work, it is not production quality, and has not been properly tested.

Benchmark

Here are some results on the Silesia compression corpus:

| File | Original | blz4 --optimal | lz4 -12 -l | lz4x -9 | | :------ | ---------: | ---------------: | -----------: | ---------: | | dickens | 10.192.446 | 4.380.430 | 4.380.430 | 4.380.430 | | mozilla | 51.220.480 | 22.025.940 | 22.025.988 | 22.025.940 | | mr | 9.970.564 | 4.190.675 | 4.190.774 | 4.190.675 | | nci | 33.553.445 | 3.621.482 | 3.621.567 | 3.621.482 | | ooffice | 6.152.192 | 3.535.258 | 3.535.258 | 3.535.258 | | osdb | 10.085.684 | 3.951.474 | 3.951.474 | 3.951.474 | | reymont | 6.627.202 | 2.063.060 | 2.063.060 | 2.063.060 | | samba | 21.606.400 | 6.100.521 | 6.100.539 | 6.100.521 | | sao | 7.251.944 | 5.668.742 | 5.668.742 | 5.668.742 | | webster | 41.458.703 | 13.835.336 | 13.835.336 | 13.835.336 | | xml | 5.345.280 | 759.868 | 759.901 | 759.868 | | x-ray | 8.474.240 | 7.177.203 | 7.177.203 | 7.177.203 |

I did not include smallz4 because it does not create output in the legacy format, so the results are not directly comparable on files larger than 4MiB.

Usage

blz4 uses Meson to generate build systems. To create one for the tools on your platform, and build blz4, use something along the lines of:

mkdir build
cd build
meson ..
ninja

You can also simply compile and link the source files.

blz4 includes the leparse and btparse algorithms from BriefLZ, which gives compression levels -5 to -9 and the very slow --optimal.

Notes

• LZ4 appears to do flexible parsing, is very close to optimal, and much faster.

Related Projects

• LZ4X
• smallz4

License

This projected is licensed under the zlib License (Zlib).