SkillAgentSearch skills...

Paq8px

PAQ8PX – Experimental Lossless Data Compressor & Entropy Estimator

Install / Use

/learn @hxim/Paq8px

README

PAQ8PX – Experimental Lossless Data Compressor & Entropy Estimator

About

PAQ is a family of experimental, high-end lossless data compression programs. paq8px is one of the longest-running branches of PAQ, started by Jan Ondrus in 2009 with major contributions from Márcio Pais and Zoltán Gotthardt (see Contribution Timeline).

paq8px consistently achieves state-of-the-art compression ratios on various data compression benchmarks (see Benchmark Results). This performance comes at the cost of speed and memory usage, which makes it impractical for production use or long-term storage. However, it is particularly well-suited for file entropy estimation and as a reference for compression research.

For detailed history and ongoing development discussions, see the paq8px thread on encode.su.

Quick start

paq8px is portable software – no installation required.

Get the latest binary for Windows (x64) from the Releases page or from the paq8px thread on encode.su, or build it from source for your platform – see below.

Command line interface

paq8px does not include a graphical user interface (GUI). All operations are performed from the command line.

Open a terminal and run paq8px with the desired options to compress your file (such as paq8px -8 file.txt).
Start with a small file – compression takes time.

Example output (on Windows):

c:\>paq8px.exe -8 file.txt
paq8px archiver v213 (c) 2026, Matt Mahoney et al.

Creating archive file.txt.paq8px213 in single file mode...

Filename: file.txt (111261 bytes)
Block segmentation:
 0           | text             |    111261 bytes [0 - 111260]
-----------------------
Total input size     : 111261
Total archive size   : 19597

Time 16.62 sec, used 2164 MB (2269587029 bytes) of memory

[!NOTE] The output archive extension is versioned (e.g., .paq8px213).

[!NOTE] You can place the binary anywhere and reference inputs/outputs by path.

Some examples

Compress a file at level 8 (balanced speed and compression ratio):

paq8px.exe -8 filename_to_compress 

Compress at the maximum level with LSTM modeling included (-12L):

paq8px.exe -12L filename_to_compress 

[!WARNING] This mode is extremely slow and memory-intensive. Make sure you have 32 GB+ RAM.

Getting help

To view available options, run paq8px without arguments. To view available options + detailed help pages, run paq8px -help.

<details> <summary>Click to expand: full <code>paq8px</code> help</summary>
paq8px archiver v213 (c) 2026, Matt Mahoney et al.
Free under GPL, http://www.gnu.org/licenses/gpl.txt

Usage:
  to compress       ->   paq8px -LEVEL[FLAGS] [OPTIONS] INPUT [OUTPUT]
  to decompress     ->   paq8px -d INPUT.paq8px213 [OUTPUT]
  to test           ->   paq8px -t INPUT.paq8px213 [OUTPUT]
  to list contents  ->   paq8px -l INPUT.paq8px213

LEVEL:
  -1 -2 -3 -4          | Compress using less memory (529, 543, 572, 630 MB)
  -5 -6 -7 -8          | Use more memory (747, 980, 1446, 2377 MB)
  -9 -10 -11 -12       | Use even more memory (4241, 7968, 15421, 29305 MB)
  -0                   | Segment and transform only, no compression
  -0L                  | Segment and transform then LSTM-only compression (alternative: -lstmonly)

FLAGS:
  L                    | Enable LSTM model (+24 MB per block type)
  A                    | Use adaptive learning rate
  S                    | Skip RGB color transform (images)
  B                    | Brute-force DEFLATE detection
  E                    | Pre-train x86/x64 model
  T                    | Pre-train text models (dictionary-based)

  Example: paq8px -8LA file.txt   <- Level 8 + LSTM + adaptive learning rate

Block detection control (compression-only):
  -forcebinary         | Force generic (binary) mode
  -forcetext           | Force text mode

LSTM-specific options (expert-only):
  -lstmlayers=N        | Set the number of LSTM layers to N (1..5, default is 2)
  -savelstm:text FILE  | Save learned LSTM model weights after compression
  -loadlstm:text FILE  | Load LSTM model weights before compression/decompression

Misc options:
  -v                   | Verbose output
  -log FILE            | Append compression results to log file
  -simd MODE           | Override SIMD detection - expert only (NONE|SSE2|AVX2|AVX512|NEON)

Notes:
  INPUT may be FILE, PATH/FILE, or @FILELIST
  OUTPUT is optional: FILE, PATH, PATH/FILE
  The archive is created in the current folder with .paq8px213 extension if OUTPUT omitted
  FLAGS are case-insensitive and only needed for compression; they may appear in any order
  INPUT must precede OUTPUT; all other OPTIONS may appear anywhere

=============
Detailed Help
=============

---------------
 1. Compression
---------------

  Compression levels control the amount of memory used during both compression and decompression.
  Higher levels generally improve compression ratio at the cost of higher memory usage and slower speed.
  Specifying the compression level is needed only for compression - no need to specify it for decompression.
  Approximately the same amount of memory will be used during compression and decompression.

  The listed memory usage for each LEVEL (-1 = 529 MB .. -12 = 29305 MB) is typical/indicative for compressing binary
  files with no preprocessing. Actual memory use is lower for text files and higher when a preprocessing step
  (segmentation and transformations) requires temporary memory. When special file types are detected, special models
  (image, jpg, audio) will be used and thus will require extra RAM.

------------------
 2. Special Levels
------------------

  -0   Only block type detection (segmentation) and block transformations are performed.
       The data is copied (verbatim or transformed); no compression happens.
       This mode is similar to a preprocessing-only tool like precomp.
       Uses approximately 3-7 MB total.

  -0L  Uses only a single LSTM model for prediction which is shared across all block types.
       Uses approximately 20-24 MB total RAM.
       Alternative: -lstmonly

---------------------
 3. Compression Flags
---------------------

  Compression flags are single-letter, case-insensitive, and appended directly to the level.
  They are valid only during compression. No need to specify them for decompression.

  L   Enable the LSTM (Long Short-Term Memory) model.
      Uses a fixed-size model, independent of compression level.

      At level -0L (also: -lstmonly) a single LSTM model is used for prediction for all detected block types.
      Block detection and segmentation are still performed, but no context mixing or Secondary Symbol
      Estimation (SSE) stage is used.

      At higher levels (-1L .. -12L) the LSTM model is included as a submodel in Context Mixing and its predictions
      are mixed with the other models.
      When special block types are detected, for each block type an individual LSTM model is created dynamically and
      used within that block type. Each such LSTM model adds approximately 24 MB to the total memory use.

  A   Enable adaptive learning rate in the CM mixer.
      May improve compression for some files.

  S   Skip RGB color transform for 24/32-bit images.
      Useful when the transform worsens compression.
      This flag has no effect when no image block types are detected.

  B   Enable brute-force DEFLATE stream detection.
      Slower but may improve detection of compressed streams.

  E   Pre-train the x86/x64 executable model.
      This option pre-trains the EXE model using the paq8px.exe binary itself.
      Archives created with a different paq8px.exe executable (even when built from the same source and build options)
      will differ. To decompress an archive created with -E, you must use the exact same executable that created it.

  T   Pre-train text-oriented models using a dictionary and expression list.
      The word list (english.dic) and expression list (english.exp) are used only to pre-train models before
      compression and they are not stored in the archive.
      You must have these same files available to decompress archives created with -T.

---------------------------
 4. Block Detection Control
---------------------------

  Block detection and segmentation always happen regardless of the memory level or other options - except when forced:

  -forcebinary

      Disable block detection; the whole file is considered as a single binary block and only the generic (binary)
      model set will be used.
      Useful when block detection produces false positives.

  -forcetext

      Disable block detection; consider the whole file as a single text block and use the text model set only.
      Useful when text data is misclassified as binary or fragments in a text file are incorrectly detected as some
      other block type.

---------------------------------------
 5. LSTM-Specific Options (expert-only)
---------------------------------------

  -lstmlayers=N

      Set the number of LSTM layers to N. Using more layers generally leads to better compression, but memory use
      will be higher (scales linearly with N) and compression time will be significantly slower. The default is N=2.

  -savelstm:text FILE

      Saves the LSTM model's learned parameters as a lossless snapshot to the specified file when compression finishes.
      Only the model used for text block(s) will be saved.
      It's not possible to save a snapshot from other block types. This is an experimental feature.

  -loadlstm:text FILE

      Loads the LSTM model's learned parameters from the specified file (which was saved earlier
      by the -savelstm:text option) before compression starts. The LSTM model will use this loaded
      snapshot to bootstrap its predictions.
      At levels -1L .. -12L only text blocks are affecte
View on GitHub
GitHub Stars255
CategoryDevelopment
Updated8h ago
Forks45

Languages

C++

Security Score

85/100

Audited on Mar 31, 2026

No findings