EasyDl

Easily download a file and save it to a local disk. It supports resuming previously downloaded files, multi-connection/parallel downloads, and retry on failure out of the box!

Features

• Resumes previous downloads, even after the program has terminated.
• Faster download speed with multiple concurrent connections.
• Automatic retry on failure.
• Supports HTTP redirects with redirect loop detection.
• No native, 100% Javascript code with zero dependency.
• Runs on Node.js, Electron, NW.js
• Easy! Dead simple API, but highly configurable when you need it.

Install

npm i -S easydl

or if you use yarn

yarn add easydl

Quickstart

EasyDl is an EventEmitter, but it also provides Promise APIs for ease of use:

try {
  const completed = await new EasyDl(
    "http://www.ovh.net/files/1Gio.dat",
    "/tmp/1GB.zip",
    { connections: 10, maxRetry: 5 }
  ).wait();
  console.log("Downloaded?", completed);
} catch (err) {
  console.log("[error]", err);
}

or if you prefer using the Promise chain:

const EasyDl = require("easydl");

new EasyDl("http://www.ovh.net/files/10Gio.dat", "~/Downloads")
  .wait()
  .then((completed) => {
    console.log("Downloaded?", completed);
  })
  .catch((err) => {
    console.log("[error]", err);
  });

• See all available options here.
• See all examples here.

---

<details> <summary> <b>Getting the Metadata</b> </summary> If you want to get the metadata of the current download, such as file size, HTTP response header, etc. You can get one by using:

try {
  const dl = new EasyDl(url, dest, options);
  const metadata = await dl.metadata();
  console.log("[metadata]", metadata);

  const success = await dl.wait();
  console.log("download complete.");
} catch (err) {
  console.log("[error]", err);
}

Alternatively, you can also listen to the metadata event:

try {
  const dl = new EasyDl(url, dest, options);
  await dl
    .on("metadata", (metadata) => {
      // do something with the metadata
    })
    .wait();
} catch (err) {
  console.log("[error]", err);
}

And you will get something like this:

{
  size: 104857600,                      // file size
  chunks: [10485760, 10485760 ....],    // size of each chunks
  isResume: true,                       // whether the current download resumes previous download (using detected chunks)
  progress: [100, 0, 100 .....],        // current progress of each chunk, values should be 0 or 100
  finalAddress: 'http://www.ovh.net/files/100Mio.dat', // final address of the file, if redirection occured, it will be different from the original url
  parallel: true,                        // whether multi-connection download is supported
  resumable: true,                      // whether the download can be stopped and resumed back later on (some servers do not support/allow it)
  headers: {
    ....
    // some http headers
    ....
  },
  savedFilePath: '/tmp/100Mio.dat'      // final file path. it may be different if you supplied directory as the dest param or you use "new_file" as the existBehavior
}

• Details of the metadata can be seen at the metadata section.
• Related example can be found here.

</details> <details> <summary> <b>Using Progress Status</b> </summary>

try {
  const dl = new EasyDl(url, dest);
  await dl
    .on("progress", ({ details, total }) => {
      // do something with the progress
      console.log("[details]", details);
      // you will get an array:
      //[
      //   { speed: 0, bytes: 0, percentage: 0 },
      //   { speed: 0, bytes: 0, percentage: 0 },
      //   ....
      //   {
      //     speed: 837509.8149186764,
      //     bytes: 7355193,
      //     percentage: 70.14458656311035
      //   },
      //   {
      //     speed: 787249.3573264781,
      //     bytes: 7507833,
      //     percentage: 71.60027503967285
      //   },
      //   ...
      //]

      console.log("[total]", total);
      // You will get:
      // {
      //   speed: 4144377.1053382815,
      //   bytes: 41647652,
      //   percentage: 39.71829605102539
      // }
    })
    .wait();
} catch (err) {
  console.log("[error]", err);
}

Note:

• details - Array containing progress information for each file chunks.
• total - The total download progress of the file.

More info: See On Progress

</details> <details> <summary> <b>Pausing/Resuming Downloads</b> </summary> EasyDl is a resilient downloader designed to survive even in the event of abrupt program termination. It can automaticaly recover the already downloaded parts of files (chunks) and resume the download instead of starting from scratch. As a result, to pause/stop the download, all you need to do is destroying the `EasyDl` instances. You will be able to resume them by creating new `EasyDl` instances later on.

try {
  const dl = new EasyDl(url, dest, opts);
  const downloaded = await dl.wait();

  // somewhere in the app you call: dl.destroy();
  // afterwards, dl.wait() will resolve and
  // this "downloaded" will be false
  console.log("[downloaded]", downloaded);
} catch (err) {
  console.log("[error]", err);
}

// later on, you decide to resume the download.
// all you need to do, is simply creating a new
// EasyDl instance with the same chunk size as before (if you're not using the default one)
try {
  // if some complete chunks of file are available, they would not be re-downloaded twice.
  const dl = new EasyDl(url, dest, opts);
  const downloaded = await dl.wait();

  console.log("[downloaded]", downloaded);
} catch (err) {
  console.log("[error]", err);
}

• More example can be found here.
• Learn more about EasyDl file chunking: Chunks.

</details> <details> <summary> <b>Without using Promise</b> </summary>

EasyDl is an EventEmitter, so if you need to, you can listen to its events instead of using the Promise APIs.

let downloaded = false;
const dl = new EasyDl(url, dest, opts)
  .on("end", () => {
    console.log("download success!");
    downloaded = true;
  })
  .on("close", () => {
    console.log(
      "this close event will be fired after when instance is stopped (destroyed)"
    );
    console.log(
      "If the download is complete (not cancelled), the .on(end) event will be fired before this event."
    );
    console.log("otherwise, only this event will be fired.");

    // downloaded will be true if .on('end') is fired
    console.log("[downloaded]", downloaded);
  })
  .on("error", (err) => {
    console.log("[error]", err);
    // handle some error here
  })
  .start(); // download will not be started unless you call .start()

</details>

CLI

<img src="https://user-images.githubusercontent.com/7076809/82736366-a0525300-9d6c-11ea-9ec5-e22dda09131f.png" alt="Demo CLI" width="600"/>

A CLI version of EasyDl is available as a separate package easydl-cli.

API

Constructor

new EasyDl(url, dest, options);

url - the URL of the file

dest - where to save the file. It can be a file name or an existing folder. If you supply a folder location (for example ~/) the file name will be derrived from the url.

options - (Object) optional configurable options:

• connections - Number of maximum parallel connections. Defaults to 5.

• existBehavior - What to do if the destination file already exists. Possible values:

  • new_file (default) - create a new file by appending (COPY) to the file name.
  • overwrite - overwrite the file. Proceed with caution.
  • error - throws error.
  • ignore - ignore and skip this download

• followRedirect - (Boolean) Whether EasyDl should follow HTTP redirection. Defaults to true

• httpOptions - Options passed to the http client. You can modify the HTTP methods, Auth, Headers, Proxy, etc here. See Node.js docs for more information.

• chunkSize - The maximum size of chunks (bytes) of the file. It accepts a fixed number or a function which let you calculate the chunk size dynamically based on the file size.

  • the default value of chunkSize is this function:

  function(size) {
      return Math.min(size / 10, 10 * 1024 * 1024);
  }
  

  • Using chunk size that is too small may lead to slower download speed due to the nature of TCP connections, but using chunk size that is too big will make resume ineffective due to many incomplete chunks.

• maxRetry - Maximum number of retries (for each chunks) when error occured. Defaults to 3.

• retryDelay - Delay in ms before attempting a retry. Defaults to 2000.

• retryBackoff - Incremental back-off in ms for each failed retries. Defaults to 3000.

• reportInterval - Set how frequent progress event emitted by EasyDL. Defaults to 2500.

• methodFallback - use GET method instead of HEAD to calculate metadata. Useful for downloading S3 signed URLs.

Metadata

You can get the metadata by using the .metadata() function or listening to the .on('metadata') event.

<details> <summary> <b>Example</b> </summary>

{
  size: 104857600,
  chunks: [10485760, 10485760 ....],
  isResume: true,
  progress: [100, 0, 100 .....],
  finalAddress: 'http://www.ovh.net/files/100Mio.dat',
  parallel: true,
  resumable: true,
  headers: {
    ....
    // some http headers
    ....
  },
  savedFilePath: '/tmp/100Mio.dat'
}

</details>

size - Size of the file in bytes. Note: If there is no information given about file size by the server, the value would be 0.

chunks - An array containing the size of eac