compressing

The missing compressing and uncompressing lib for node.

Currently supported:

• tar
• gzip
• tgz
• zip

Install

npm install compressing

Usage

Compress a single file

Use gzip as an example, tar, tgz and zip is same as gzip.

promise style

const compressing = require('compressing');

// compress a file
compressing.gzip.compressFile('file/path/to/compress', 'path/to/destination.gz')
.then(compressDone)
.catch(handleError);

// compress a file buffer
compressing.gzip.compressFile(buffer, 'path/to/destination.gz')
.then(compressDone)
.catch(handleError);

// compress a stream
compressing.gzip.compressFile(stream, 'path/to/destination.gz')
.then(compressDone)
.catch(handleError);

stream style

const compressing = require('compressing');

new compressing.gzip.FileStream({ source: 'file/path/to/compress' })
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/destination.gz'))
  .on('error', handleError);

// It's a transform stream, so you can pipe to it
fs.createReadStream('file/path/to/compress')
  .on('error', handleError)
  .pipe(new compressing.gzip.FileStream())
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/destination.gz'))
  .on('error', handleError);

// You should take care of stream errors in caution, use pump to handle error in one place
const { pipeline: pump } = require('stream');
const sourceStream = fs.createReadStream('file/path/to/compress');
const gzipStream = new compressing.gzip.FileStream();
const destStream = fs.createWriteStream('path/to/destination.gz');
pump(sourceStream, gzipStream, destStream, handleError);

Compress a dir

Use tar as an example, tgz and zip is same as gzip.

Gzip only support compressing a single file. if you want to compress a dir with gzip, then you may need tgz instead.

promise style

const compressing = require('compressing');
compressing.tar.compressDir('dir/path/to/compress', 'path/to/destination.tar')
.then(compressDone)
.catch(handleError);

stream style

const compressing = require('compressing');

const tarStream = new compressing.tar.Stream();
tarStream.addEntry('dir/path/to/compress');

tarStream
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/destination.tar'))
  .on('error', handleError);

// You should take care of stream errors in caution, use pump to handle error in one place
const tarStream = new compressing.tar.Stream();
tarStream.addEntry('dir/path/to/compress');
const destStream = fs.createWriteStream('path/to/destination.tar');
pump(tarStream, destStream, handleError);

Stream is very powerful, you can compress multiple entries in it;

const tarStream = new compressing.tar.Stream();
// dir
tarStream.addEntry('dir/path/to/compress');

// file
tarStream.addEntry('file/path/to/compress');

// buffer
tarStream.addEntry(buffer);

// stream
tarStream.addEntry(stream);

const destStream = fs.createWriteStream('path/to/destination.tar');
pipe(tarStream, destStream, handleError);

Uncompress a file

promise style

const compressing = require('compressing');

// uncompress a file
compressing.tgz.uncompress('file/path/to/uncompress.tgz', 'path/to/destination/dir')
.then(uncompressDone)
.catch(handleError);

// uncompress a file buffer
compressing.tgz.uncompress(buffer, 'path/to/destination/dir')
.then(uncompressDone)
.catch(handleError);

// uncompress a stream
compressing.tgz.uncompress(stream, 'path/to/destination/dir')
.then(uncompressDone)
.catch(handleError);

Note: tar, tgz and zip have the same uncompressing API as above: destination should be a path of a directory, while that of gzip is slightly different: destination must be a file or filestream.

And working with urllib is super easy. Let's download a tgz file and uncompress to a directory:

const urllib = require('urllib');
const targetDir = require('os').tmpdir();
const compressing = require('compressing');

urllib.request('http://registry.npmjs.org/compressing/-/compressing-2.0.0.tgz', {
  streaming: true,
  followRedirect: true,
})
.then(result => compressing.tgz.uncompress(result.res, targetDir))
.then(() => console.log('uncompress done'))
.catch(console.error);

stream style

const compressing = require('compressing');
const mkdirp = require('mkdirp');

function onEntry(header, stream, next) => {
  stream.on('end', next);

  // header.type => file | directory
  // header.name => path name

  if (header.type === 'file') {
    stream.pipe(fs.createWriteStream(path.join(destDir, header.name)));
  } else { // directory
    fs.mkdir(path.join(destDir, header.name), { recursive: true }, err => {
      if (err) return handleError(err);
      stream.resume();
    });
  }
}

new compressing.tgz.UncompressStream({ source: 'file/path/to/uncompress.tgz' })
  .on('error', handleError)
  .on('finish', handleFinish) // uncompressing is done
  .on('entry', onEntry);

// It's a writable stream, so you can pipe to it
fs.createReadStream('file/path/to/uncompress')
  .on('error', handleError)
  .pipe(new compressing.tgz.UncompressStream())
  .on('error', handleError)
  .on('finish', handleFinish) // uncompressing is done
  .on('entry', onEntry);

Note: tar, tgz and zip have the same uncompressing streaming API as above: it's a writable stream, and entries will be emitted while uncompressing one after one another, while that of gzip is slightly different: gzip.UncompressStream is a transform stream, so no entry event will be emitted and you can just pipe to another stream

This constrants is brought by Gzip algorithm itself, it only support compressing one file and uncompress one file.

new compressing.gzip.UncompressStream({ source: 'file/path/to/uncompress.gz' })
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/dest/file'))
  .on('error', handleError);

API

compressFile

Use this API to compress a single file. This is a convenient method, which wraps FileStream API below, but you can handle error in one place.

• gzip.compressFile(source, dest, opts)
• tar.compressFile(source, dest, opts)
• tgz.compressFile(source, dest, opts)
• zip.compressFile(source, dest, opts)

Params

• source {String|Buffer|Stream} - source to be compressed, could be a file path, buffer, or a readable stream
• dest {String|Stream} - compressing destination, could be a file path(eg. /path/to/xx.tgz), or a writable stream.
• opts {Object} - usually you don't need it

Returns a promise object.

compressDir

Use this API to compress a dir. This is a convenient method, which wraps Stream API below, but you can handle error in one place.

Note: gzip do not have a compressDir method, you may need tgz instead.

• tar.compressDir(source, dest, opts)
• tgz.compressDir(source, dest, opts)
• zip.compressDir(source, dest, opts)

Params

• source {String|Buffer|Stream} - source to be compressed
• dest {String|Stream} - compressing destination, could be a file path(eg. /path/to/xx.tgz), or a writable stream.
• opts {Object} - usually you don't need it

uncompress

Use this API to uncompress a file. This is a convenient method, which wraps UncompressStream API below, but you can handle error in one place. RECOMMANDED.

• tar.uncompress(source, dest, opts)
• tgz.uncompress(source, dest, opts)
• zip.uncompress(source, dest, opts)
• gzip.uncompress(source, dest, opts)

Params

• source {String|Buffer|Stream} - source to be uncompressed
• dest {String|Stream} - uncompressing destination. When uncompressing tar, tgz and zip, it should be a directory path (eg. /path/to/xx). When uncompressing gzip, it should be a file path or a writable stream.
• opts {Object} - usually you don't need it

  • opts.zipFileNameEncoding {String} - Only work on zip format, default is 'utf8'. Major non-UTF8 encodings by languages:

    • Korean: cp949, euc-kr
    • Japanese: sjis (shift_jis), cp932, euc-jp
    • Chinese: gbk, gb18030, gb2312, cp936, hkscs, big5, cp950

  • opts.strip {Number} - Strip leading path segments when extracting (tar/tgz/zip). Default is 0.

FileStream

The transform stream to compress a single file.

Note: If you are not very familiar with streams, just use compressFile() API, error can be handled in one place.

• new gzip.FileStream(opts)
• new tar.FileStream(opts)
• new tgz.FileStream(opts)
• new zip.FileStream(opts)

Common params:

• opts.source {String|Buffer|Stream} - source to be compressed, could be a file path, buffer, or a readable stream.

Gzip params:

• opts.zlib - {Object} gzip.FileStream uses zlib to compress, pass this param to control the behavior of zlib.

Tar params:

• opts.relativePath {String} - Adds a file from source into the compressed result file as opts.relativePath. Uncompression programs would extract the file from the compressed file as relativePath. If opts.source is a file path, opts.relativePath is optional, otherwise it's required.
• opts.size {Number} - Tar compression requires the size of file in advance. When opts.source is a stream, the size of it cannot be calcu